From 492c4a8867992097cf959f75dafc4badbb228d7e Mon Sep 17 00:00:00 2001 From: Matthew Huntington Date: Tue, 24 Oct 2023 19:37:18 -0500 Subject: [PATCH] better descriptions for sqlite3 --- python_sql/lessons/python_sql.md | 83 +++++++++++++++++++++----------- 1 file changed, 54 insertions(+), 29 deletions(-) diff --git a/python_sql/lessons/python_sql.md b/python_sql/lessons/python_sql.md index cc543f2..195cc96 100644 --- a/python_sql/lessons/python_sql.md +++ b/python_sql/lessons/python_sql.md @@ -13,19 +13,36 @@ ## Install a Python virtual environment +A virtual environment allows us to create spaces that hold all of the dependencies need we for our various projects. We can create as many as we like, wherever we like, and each virtual environment van be used for as many projects as we'd like. + +Let's create one: + ```python python3 -m venv ~/my-env +``` + +Once created, we can activate it like so: + +```python source ~/my-env/bin/activate ``` +Once activated, any python packages that we install will be installed just for that virtual environment. We can switch between virtual environments simply by activating them, thus allowing us to have different sets of dependencies available to us for different projects. + ## Install psychopg2-binary +The `psychopg2-binary` Python package facilitates communication between Python and PostgrSQL. It's not necessary in order to do so, but it makes our lives *much* easier. + ``` python -m pip install psycopg2-binary ``` ## Connect to Postgres via Python +Once we've installed `psychopg2-binary`, we can import it just like any other package that comes as part of the default Python installation. + +To connect to the database, we need to run `psycopg2.connect()` and supply the correct data. To close the connection, run `conn.close()` + ```python import psycopg2 conn = psycopg2.connect( @@ -38,7 +55,9 @@ conn.close() ## Running Queries with Python -Select many +Running queries requires a "cursor" which is basically just something that performs operations on the database for us. + +Let's create a cursor and then have it perform a `SELECT` query. Once, the cursor has executed the query, it gives us a few functions that control have we view the data. Let's fetch all of the rows returned from the execution of the query and place them in a list. Note that each row is represented as a `tuple` is just an immutable list. When you're done, you'll need to close the cursor with `cursor.close()` in addition to closing the connection (shown above) ```python cursor = conn.cursor() @@ -50,28 +69,28 @@ print(cursor.fetchall()) cursor.close() ``` -Select one +Now let's do a `SELECT` query, but let's also pass a value into the the query. The `%s` below is replaced with the number `24` (I'll explain the list syntax in the next section). This helps us prevent SQL injection. `fetchone` returns just the single row, as a tuple (not inside a list). Note that even if the `SELECT` statement had returned multiple rows, `fetchone` would give us only the first row as a tuple (not inside a list). ```python cursor.execute("SELECT * FROM people WHERE id = %s", [24]) print(cursor.fetchone()) ``` -Insert +Now let's run an `INSERT` with two parameters passed in. The order of the `%s` placeholders must match the order of the values in the list parameter. The `conn.commit()` will write the changes to disk, giving us an opportunity to review what we've done or perform additional operations before they become permanent. ```python cursor.execute("INSERT INTO people (name, age) VALUES (%s, %s)", ['Matt', 43]) conn.commit() ``` -Delete +Delete is pretty simple once we know the basics ```python cursor.execute("DELETE FROM people WHERE id = %s", [24]); conn.commit() ``` -Update +Update is just more of the same, but make sure your array parameter matches the `%s` placeholders ```python cursor.execute("UPDATE people SET name = %s, age = %s WHERE id = %s", ['Matt', 43, 20]) @@ -80,7 +99,7 @@ conn.commit() ## SQLite Basics -Connect: +SQLite functionality comes as part of the default python installation (no packages to install!) and is very similar to `psycopg2-binary`. Note that `sqlite3.connect()` takes as a parameter the location where the database file is stored. ```python import sqlite3 @@ -90,49 +109,49 @@ con = sqlite3.connect("mydb.db") con.close() ``` -Create table +Create a table (this could also be done in `psycopg2-binary` as well). Note there is no need to close the cursor ```python cur = con.cursor() cur.execute("CREATE TABLE people (name, age)") ``` -Select Many +Very similar, except that `cur.execute` returns a result set that has the `fetchone`/`fetchall` functionality ```python res = cur.execute("SELECT * FROM people") print(res.fetchall()) ``` -Select One +Selecting a single row is similar to `psycopg2-binary`, but you use `?` instead of `%s` ```python -res = cur.execute("SELECT * FROM people WHERE name = 'Matt'") +res = cur.execute("SELECT * FROM people WHERE name = ?", ['Matt']) print(res.fetchone()) ``` -Insert +`INSERT` works as expected ```python cur.execute("INSERT INTO people (name, age) VALUES (?, ?)", ['Zagthorp', 543]) con.commit() ``` -Delete +So does `DELETE` ```python cur.execute("DELETE FROM people WHERE name = ?", ['Bilbo']) con.commit() ``` -Update +and `UPDATE` ```python cur.execute("UPDATE people SET name = ? WHERE name = ?", ['Bilbo', 'Matthew']) con.commit() ``` -Check via `sqlite3` in the terminal +At any point, you can use the `sqlite3` terminal command to interface with the database, just like you would with `psql` ``` sqlite3 mydb.db @@ -140,6 +159,8 @@ sqlite3 mydb.db ## Migrating data to a SQLite +At this point, migrating data from PostgreSQL to SQLite is just a matter of putting together what we've learned + ```python # PostgreSQL @@ -172,7 +193,7 @@ con.close() ## CSV Basics -mydb.csv: +Let's create a basic CSV file that we'll import into PostgreSQL ```csv 1,matt,43 @@ -180,28 +201,31 @@ mydb.csv: 3,zagthorp,999 ``` -read: +Like SQLite, CSV functionality comes with Python. First, `open()` (a global function in Python) the file you want to read. Pass the `csvfile` reference variable into `csv.reader()` to get the representation of the CSV file in Python (basically a list of lists). Loop/print and then close the file with `csvfile.close()` ```python import csv -with open('mydb.csv', newline='') as csvfile: - people = csv.reader(csvfile, delimiter=',') - for person in people: - print(person) +csvfile = open('mydb.csv') +people = csv.reader(csvfile) +for person in people: + print(person) +csvfile.close() ``` -wrte: +Writing is pretty similar, except you need to pas `'w'` into `open()` as a second parameter. Use `csv.writer()` to convert lists to CSV notation, and `writer.writerow()` to save a row to the file. Don't forget to close the file! ```python -import csv -with open('writedb.csv', 'w') as csvfile: - writer = csv.writer(csvfile) - writer.writerow([1, 'matt', 43]) +csvfile = open('mydb.csv', 'w') +writer = csv.writer(csvfile) +writer.writerow([1, 'matt', 43]) +csvfile.close() ``` ## Exporting data to CSV +This is very similar to how we exported to SQLite. The PostgreSQL section doesn't change. + ```python # PostgreSQL @@ -222,8 +246,9 @@ conn.close() import csv -with open('writedb.csv', 'w') as csvfile: - writer = csv.writer(csvfile) - for person in people: - writer.writerow(person) +csvfile = open('mydb.csv', 'w') +writer = csv.writer(csvfile) +for person in people: + writer.writerow(person) +csvfile.close() ```