An important advantage of database management systems is Data Independence.
Data Independence refers to the insulation from applications programs from changes in the way the data is structured and stored.
Levels of Abstraction in a DBMS
graph TD F(fa:fa-table View 1)---E G(fa:fa-table View 2)---E H(fa:fa-table View 3)---E["fa:fa-database Conceptual Schema"] E---A["fa:fa-server Physical Schema"] A---B["fa:fa-HDD-o Disk"] Logical Independence If the structure of the data changes, applications and users can be insulated from this changes thanks to external schemas (in SQL, views).
Usage of the Pandas quantile function to analyze Fortune500 data. Data visualizations using Seaborn
In Jupyter Lab, the %sql magic command persist creates a new table in the database to which we are connected. The table name will be the name as the name of the Python variable.
Example:
import pandas as pd df = pd.read_csv('/tmp/data.csv') df column1 column2 0 1 a 1 2 b 2 3 b # set environment variable $DATABASE_URL %sql postgresql://datacomy:PASSWORD@localhost/datacomy 'Connected: datacomy@datacomy' %sql persist df * postgresql://datacomy:***@localhost/datacomy 'Persisted df' %%sql SELECT tablename,tableowner FROM pg_catalog.
To permanently include packages or folder into the PYTHONPATH of an Anaconda environment, activate the Conda environment and use [conda develop](https://docs.conda.io/projects/conda-build/en/latest/resources/commands/conda-develop.html) to add the path permanently to the PYTHONPATH of the Conda environment.
conda develop /PATH/TO/YOUR/FOLDER/WITH/MODULES This command will create file named conda.pth file the site-packages folder of your environment.
~/anaconda3/envs/YOUR_ENV/lib/pythonX.X/site-packages
As an alternative, you can just add a conda.pth file in ~/anaconda3/envs/YOUR_ENV/lib/pythonX.X/site-packages with the folder that you want to include in your PYTHONPATH.
To change the character limit for the Black Python formatter, you can add a file named [pyproject.toml](https://www.python.org/dev/peps/pep-0518/) with the following:
[tool.black] line-length = 80 in your project directory.
This change is permanent but will work only in the project with the mentioned file. As of July 2020, I haven’t found a way to make this change permanent globally to a Python environment
How to Slice Numpy Arrays
Combining Data with Pandas - Merge - Inner, Outer, Left and Right joins
Bisection Search Algorithm in Python
Regular expressions, usually called regex are sequence of characters that define a search pattern.
Regular expressions are extemely useful for findind strings or for input validation.
In this article, we will learn how to use regular expressions and implement the in Python.