Main concepts of HTTP:
Try to run the HTTP protocol by hand at the command line:
$ nc -c example.com 80 GET /index.html HTTP/1.1 Host: example.com HTTP/1.1 200 OK Accept-Ranges: bytes ...
Notice that the response contains some HTML:
<!doctype html> <html> <head> <title>Example Domain</title> ...
And that also includes an <a> tag with
another URL in it:
...
<body>
<div>
<h1>Example Domain</h1>
<p>This domain is for use in illustrative examples in documents. You may use this
domain in literature without prior coordination or asking for permission.</p>
<p><a href="https://www.iana.org/domains/example">More information...</a></p>
...
What a browser does is automate this entire HTTP process for you. You type a URL into your browser's address bar and your browser: opens a network connection to the server, sends a textual request to the server as above in the format defined by the HTTP protocol, receives the HTTP response which contains HTML, displays that HTML for you visually, and allows you to click on links, which would repeat the process.
What we have just seen here is how you can partly do this process yourself manually. What we will be doing today is using Python and Processing code to automate this process. Not to fetch web pages to display them visually to a user, but to fetch JSON code that you will use to render something within Processing.
So in a way, it is like Processing is acting as a non-traditional browser, and we have some Python code fetching data to display.
Start a new sketch with this Processing code:
(Note: if you were already attempting this based on last week's class notes, start fresh with teh code linked below. I made some changes to debug some issues with the code from last week.)
As we discussed last week, make a new tab (you will need to save
your sketch first), name it week13_networking, add
copy/paste the following code into it:
Edit that tab to change the IP address to my computer's IP
address, and change the port to 8080.
Now (if my server is running!) you should be able to run your code to see your own "creature" as well as that of everyone else.
Note the lines
in week13_networking.pyde that look
like this:
if millis() > lastGet + 1000:
lastGet = millis()
As the comment there explains, this code limits you to only
fetching new data once per second, even if your code tries to
fetch it more frequently. The framerate of your
sketch is probably somewhere in the range of 30-60 frames per
second. And the way
week13_creatures_net.pyde is
implemented, it tried to fetch data once-per-frame. That would
potentially be a lot of network traffic. Especially if everyone
in our class was running this at the same time. This limit
prevents that excessive network traffic from happening. This
technique is often called throttling. (I used
to think "throttling" meant to do more / faster / bigger because
of the expression "full throttle" meaning to go fast. But
actually, "to throttle" means to limit something, like the output
of an engine. So full throttle means no limitation.)
Processing is rendering your local version which is moving very quickly, as well as the server's version of your creature, which is slower due to throttling.
To remedy the appearance of this, try adding this line:
i = 0
if creatureList:
while i < len(creatureList):
creature = creatureList[i]
if creature["name"] != myCreature["name"]:
fill( creature["color"] )
rect(creature["x"],creature["y"],creature["size"],creature["size"])
i = i + 1
Here is the code:
Let's talk about it.
Thought experiment: would it work if the server simply saved all the data from individual users as JSON in one file? What might go wrong?
Solution: ACID compliance (Atomicity, Consistency, Isolation, and Durability). Making sure that concurrent requests to save or access data are not interleaved and do not overwrite each other.
These features are implemented with a technology called a database. There are many popular examples: Oracle, Postgres, MySQL. You may have heard of newer database technologies like MongoDB, Redis, or CouchDB.
A simple database: Sqlite
You can run Sqlite from the command line by
typing sqlite3. But this won't do
much unless you pass it an argument, which is the name of a
file. Turns out that even databases store their data in files,
like we were doing with Python's basic file IO (read and
write) mechanisms earlier, but database programs manage access
to these files in ways that are ACID compliant (or at least,
partially so).
So, let's run Sqlite by specifying a temporary database file for some experimentation:
$ sqlite3 test.sqlite SQLite version 3.39.5 2022-10-14 20:58:05 Enter ".help" for usage hints. sqlite>
.sqlite is a customary file extension
for Sqlite database files.
Now we are in a Sqlite shell and can type various Sqlite commands, which are mainly in the SQL language, which stands for Structure Query Language.
sqlite> .tables sqlite>
.tables lists
all tables in your database, but we
haven't created any yet, so this returns nothing.
You can think of tables as like spreadsheets. A table is basically one page on Google Sheets. You can define all the columns of the database, then add rows, and then query the database by retrieving rows with certain criteria. Here are some commands to do those operations:
sqlite> create table creatures (name TEXT, size INTEGER);
sqlite> .tables
creatures
sqlite> insert into creatures (name, size) values ("Rory", 50);
sqlite> insert into creatures (name, size) values ("Gritty", 100);
sqlite> select * from creatures;
Rory|50
Gritty|100
Hopefully you can see how these various technologies and techniques that we have been learning in recent weeks can work well together: databases, dictionaries, other datastructures, and the JSON used to serialize them.
What is flask?
We could simplify this server code and interact with it manually
using nc.