Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
**/models/


# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
Expand Down
136 changes: 136 additions & 0 deletions requirements_mac.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
absl-py==1.3.0
addict==2.4.0
anyio==3.6.2
appnope==0.1.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.1.0
astunparse==1.6.3
attrs==22.1.0
backcall==0.2.0
beautifulsoup4==4.11.1
black==22.10.0
bleach==5.0.1
cachetools==5.2.0
certifi==2022.9.24
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
contourpy==1.0.6
cycler==0.11.0
debugpy==1.6.3
decorator==5.1.1
defusedxml==0.7.1
entrypoints==0.4
executing==1.2.0
fastapi==0.85.1
fastjsonschema==2.16.2
flatbuffers==1.12
fonttools==4.38.0
gast==0.4.0
google-auth==2.14.1
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.50.0
h11==0.14.0
h5py==3.7.0
idna==3.4
IProgress==0.4
ipykernel==6.17.0
ipython==8.6.0
ipython-genutils==0.2.0
ipywidgets==8.0.2
jedi==0.18.1
Jinja2==3.1.2
jsonschema==4.16.0
jstyleson==0.0.2
jupyter-server==1.21.0
jupyter_client==7.4.4
jupyter_core==4.11.2
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.3
keras==2.9.0
Keras-Preprocessing==1.1.2
kiwisolver==1.4.4
libclang==14.0.6
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib==3.6.1
matplotlib-inline==0.1.6
mistune==2.0.4
mypy-extensions==0.4.3
nbclassic==0.4.7
nbclient==0.7.0
nbconvert==7.2.3
nbformat==5.7.0
nest-asyncio==1.5.6
networkx==2.8
notebook==6.5.2
notebook_shim==0.2.0
numpy==1.23.1
oauthlib==3.2.2
opencv-python==4.6.0.66
openvino @ file:///Users/chepe/hackerdojo/pythonista/openvino/build/wheels/openvino-2022.3.0-000-cp310-cp310-macosx_12_0_arm64.whl
openvino-dev @ file:///Users/chepe/hackerdojo/pythonista/openvino/build/wheels/openvino_dev-2022.3.0-000-py3-none-any.whl
openvino-telemetry==2022.1.1
opt-einsum==3.3.0
packaging==21.3
pandas==1.1.5
pandocfilters==1.5.0
parso==0.8.3
pathspec==0.10.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.3.0
platformdirs==2.5.2
progress==1.6
prometheus-client==0.15.0
prompt-toolkit==3.0.31
protobuf==3.19.6
psutil==5.9.3
ptyprocess==0.7.0
pure-eval==0.2.2
py-cpuinfo==9.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pydantic==1.10.2
Pygments==2.13.0
pyparsing==3.0.9
pyrsistent==0.19.1
python-dateutil==2.8.2
pytz==2022.6
PyYAML==6.0
pyzmq==24.0.1
requests==2.28.1
requests-oauthlib==1.3.1
rsa==4.9
scipy==1.9.3
Send2Trash==1.8.0
six==1.16.0
sniffio==1.3.0
soupsieve==2.3.2.post1
stack-data==0.6.0
starlette==0.20.4
tensorboard==2.9.1
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow-estimator==2.9.0
tensorflow-macos==2.9.2
termcolor==2.1.0
terminado==0.17.0
texttable==1.6.4
tinycss2==1.2.1
tomli==2.0.1
tornado==6.2
tqdm==4.64.1
traitlets==5.5.0
typing_extensions==4.4.0
urllib3==1.26.12
uvicorn==0.18.3
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.4.1
Werkzeug==2.2.2
widgetsnbextension==4.0.3
wrapt==1.14.1
2 changes: 1 addition & 1 deletion session_1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ python3 -m venv ~/.virtualenvs/hd-python

Activate the virtual environment (Linux & Mac):
```
source ~/.virtualenv/hd-pythn/bin/activate
source ~/.virtualenvs/hd-python/bin/activate
```
or run the script (Windows only):
```
Expand Down
148 changes: 141 additions & 7 deletions session_2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,15 @@ If this is your first time developing with python, follow the steps below:
- About [Black](https://black.readthedocs.io/en/stable/) formatter and why its awesome.
- Have a virtual environment already configured as an interpreter for your project.
- For this, you can try completing all the steps from [Session 1](../session_1/README.md).
- Activate your virtual envirnoment and install Jupyter Notebook.

```
source ~/.virtualenvs/hd-python/bin/activate
pip install notebook
```
- Open VS Code and make sure you are using the interpreter from your virtual environment.
- Install the Jupyter VS Code extension (by Microsoft).



## Overview of the building blocks
Expand All @@ -32,7 +41,6 @@ Some of the concepts we will glance over are:
- Functions.
- Classes and magic *dunder* methods (double underscore methods).
- Modules, packages and the import system.
- A few, very useful, *advanced-ish* concepts (context managers, decorators, iterators/generators)

Depending on the type of a variable, it will behave differently when used in combination with the different operators that exist in python. You can take a look at al the operators available here:

Expand All @@ -51,7 +59,20 @@ There are a few more types available, and you may find an interesting read at th
- Dict (`dict`)
- String (`str`)

Please take a look at the [Session 2 Notebook](./Session_2.ipynb) to take a look at interesting things we can do with each type.
### Modern Python: Type hints

It is a fact that a trend in the industry is to provide some typing information as we write code. This helps us prevent and catch errors before we actually run our code, as well as (sometimes) make it execute faster (ie. typescript vs javascript) or provide functionality that would be very hard to implement without typing information.

Since Python 3.5, you can add type hints to your code by importing the definitions in the [typing](https://docs.python.org/3/library/typing.html) module. Starting on Python 3.10, there is no need to import anything as it is becoming a fundamental part of the language.

I recommend that you enable your linter to warn you about typing information and use them as much as possible. To do this, go to the settings page in VS Code (Mac: `cmd + ,` Win,Linux: `crtl + ,`), write `type checking` and select the corresponding option in the

Personally, I have a `strict` setting for my projects, but if you just want to get your feet wet and see what this type hints are all about you may start with the `basic` setting.

![Alt text](./assets/type_checking.png "Enable type checking")

Please take a look at the [Session 2 Notebook](./Session_2.ipynb) to take a look at interesting things we can do with each type and the corresponding hints.


## Scopes and lifetimes

Expand Down Expand Up @@ -132,22 +153,135 @@ my_func(a=1, 2)

### Pitfalls: Mutable types as default values (don't do it)

When you run a Python script or import it as a module, all the top level statements will be evaluated. That includes any global variable initialization, function calls and function definitions. As everything else in Python, a function is an object that exists during the lifetime of a program, and as any other object, some of it's members have values assigned. This is the case of the default values for `keyword` arguments.

For most default values, this is great: whenever someone invokes a function without passing a value, the default will be used. The problem arises when the type of this default value is a mutable type (list, dict, etc) and said value is modified inside the function. The next time that function is called, the default value will be different from the first time:

```
def func(key, value, d = {}):
d[key] = value
return d
```

When we call it the first time, the result is what we expect:

```
func('first', 1)
```
this outputs `{'first': 1}`

And when we call it the second time:
```
func('second', 2)
```

We would expect to see `{'second': 2}`, however, what we get is:

```
{
'first': 1,
'second': 2
}
```

This behavior is a consequence of the fudamental design of the Python language and, arguably, can be a *useful* feature (ie. memoization). However, there are much better way to implement this kind of behaviors whenever we intend to.

If you need to use a mutable type for a function, my recommendation is that you use an invalid value as the default and then assign it to the variable when the function execution starts. A common idiom for this very common case is a shorthand for:

```
if d == None
d = {}
```
has an equivalent as:

```
d = d or {}
```

So now, our function looks like this:

```
def func(key, value, d = None):
d = d or {}

d[key] = value
return d
```

and after calling it 2 times, the result is what we expect:

```
func('first', 1)
func('second', 2)
```
Now the result is the expected `{'second': 2}`

## Classes

A class allow us to define our own types. The syntax is very simple and similar to other languages. It allows us to group together several related functions and data. Some people like OOP, others prefer using only functions. The good thing is that Python allows you to use one or the other (or both) whenever you want.

For this section, take a look at the [notebook](./Session_2.ipynb).

## Modules and the import system

The import system is a Python component that allows us to package (and publish) fractions of our code. A module is the next level of grouping after functions and classes
### Magic *dunder* methods

A *dunder* method (special methods or Double Underscore Method) is a method defined inside a class with [special behavior](https://docs.python.org/3/reference/datamodel.html#specialnames), suche methods' name starts and ends with `__` (double underscore, thus the *dunder* moniker).

There's a special magic that can happen when a certain method is implemented: It allows us to use standard features of Python with our custom objects. There are many *dunder* methods available, but some of the ones I find very useful are:

- Initialization: `__init__`
- String representation methods: `__str__` and `__repr__`
- Hash: `__hash__`
- Sorting and comparison: `__lt__`, `__lte__`, `__gt__`, `__gte__`

### Hack: Get free out-of-the-box functionality with `dataclass`

The `dataclass` decorator is part of the standard library (meaning: you have it already) and provides automatically generated *dunder* methods that make your life easy and save you precious developer time.

One feature that I find particularly useful is the ability to compare (and sort) objects out of the box, based on the existing members.

Check out the [Official Documentation for dataclasses](https://docs.python.org/3/library/dataclasses.html) and also the examples provided in the [notebook](./Session_2.ipynb).

### Better Hack: Pydantic

[Pydantic](https://pydantic-docs.helpmanual.io) is a library that takes these concepts to the next level. It has an implementation for dataclasses that is a drop-in replacement for the standard library but also provide a `BaseModel` type that is great for modeling and validating data. A lot of the [FastAPI](https://fastapi.tiangolo.com) functionality introduced in [Sesion 1](../session_1/README.md) is built using Pydantic to provide the magic. We will take a deeper look at pydantic in the 3rd session.


## Modules, packages and the import system

The import system is a Python component that allows us to package (and publish) fractions of our code. A module is the next level of grouping after functions and classes as it allows the programmer to put multiple class definitions and functionality together in a file. And what do we get if we create a directory and put several modules in there? That would be a `package` and as the name suggests, with just a few extra steps it can be packaged for publishing and distribution.

It is important to know that every time the interpreter runs an `import` statement, all the top level lines will run (as we saw on the `functions` section). But also, if we do the same import in multiple places in our program, it will only run once (under most circumstances).

If we want our module to be able to run as a script, we can use the following lines (that I'm sure you've seen if this is not your first day playing with Python):

```
if __name__ == "__main__":
# Do stuff when invoked with `python myscript.py`
...
```

When a Python module or package is imported, `__name__` is set to the module’s name. Usually, this is the name of the Python file itself without the .py extension. However, if the module is executed in the top-level code environment, its __name__ is set to the string '__main__'. This way, we can prevent some of the code from running when using the file as a module, but execute some special things for us when invoked as top-level by the interpreter.

Depending on the version of Python you are using, you might need to include a `__init__.py` file as part of your package. I recommend that you always create this file. In many cases this item can be empty and is just used as a marker, but if your package is more complex, you might want to isolate the user of your library from some parts of your code and provide a more useful interface for them to use.

So, in our project, we might have a combination of modules and libraries that looks like this:

```
project_folder
| main.py # Entry point
|-firstmodule.py # Some functionality in a module
|-mypackage # Some extra functionality in a package
|-__init__.py # Marker file
|-secondmodule.py # Actual implementations
```


## Bonus: Context Managers, Generators, Decorators
## Putting it all together: Custom inferencing library using ***Intel's OpenVINO***

A brief summary of these very useful concepts.
We will define a single class `Witi` (What is this image?) that loads a pre trained model when we create an instance so we can call a `predict` method and get a description for an image we give it.

## Putting it all together: Inferencing library using Intel's OpenVINO
As always, take a look at the [notebook](./Session_2.ipynb) to see how this is done and also look at the code in this repo.

## Thanks to Intel

Expand Down
Loading