From 8c1bdeeb0b18d134d2fec684789be55d03bfc482 Mon Sep 17 00:00:00 2001 From: Roberto Vega Date: Fri, 25 Aug 2023 03:05:53 +0100 Subject: [PATCH 01/12] Removed point 4 on code_quality README to include it as an exercise on assignment_2. Added a second part of assignment_2 focused on project structure. --- 05_assessing_code_quality/README.md | 2 +- assignments/assignment_2/README.md | 22 ++++++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/05_assessing_code_quality/README.md b/05_assessing_code_quality/README.md index f516a32..320dc30 100644 --- a/05_assessing_code_quality/README.md +++ b/05_assessing_code_quality/README.md @@ -9,7 +9,7 @@ Our goal here is to equip you with the skills to critically evaluate other peopl 1. Read about [Project Structure](./1_project_structure.md) and learn how to organize your codebase. A well-organized codebase makes understanding and reviewing code a breeze. 2. These practical [Code Review Guidelines](./2_code_reviews.md) will help you give feedback in a code review. 3. Read about some of the [Code Smells](./3_code_smells.md) to be aware of during code reviews. That will help you sniff out bad practices and improve code quality. -4. For the next weekly session, you will be asked to review the structure of an existing project. Choose a project from your team and review its organization. What would you change? What would you keep? Why? + ## Assignment #2 diff --git a/assignments/assignment_2/README.md b/assignments/assignment_2/README.md index a8099bf..d4b95b4 100644 --- a/assignments/assignment_2/README.md +++ b/assignments/assignment_2/README.md @@ -48,3 +48,25 @@ The reviewer will only have access to code that's different from the `main` bran 3. Assign a colleague to review your code. 4. After the review, merge the pull request. 5. Finally a pull request from `refactoring` to `main` or simply merge the `refactoring`. + + +# Assignment 2: Project Structure + +We know that the things presented in the previous modules are orientative guidelines and real-life scenarios will slightly differ based on the project's specific needs. However, it is not uncommon to unnecessarily deviate too much from these. + +## 1- Project restructuration + +This exercise is very simple, we just want you to fish out some current or older project that you are or were part of and review its organization. What would you change? What would you keep? Why? + + +## 2- Structure recognition + +In collecting a couple of examples for the project structure module we made a mess and lost track of the structure each of the projects. Give it a go and see if you can figure out which patterns do these projects follow. + +[Link to mysterious project 1](https://github.com/qiuyu96/CoDeF/tree/main) + +[Link to mysterious project 2](https://github.com/RoberVega/mysterious_project_2) + +[Link to mysterious project 3](https://github.com/RoberVega/mysterious_project_3) + +[Link to mysterious project 4](https://github.com/cookiecutter/cookiecutter/tree/main) From fe2bc05c029750b18d9fe6f7a29eafc0a800d0d5 Mon Sep 17 00:00:00 2001 From: Roberto Vega Date: Fri, 25 Aug 2023 16:30:44 +0100 Subject: [PATCH 02/12] Updated assignments and a part of the life_expectancy files --- README.md | 4 +- assignments/assignment_3/README.md | 51 +++++++++++++++---- assignments/assignment_4/README.md | 2 +- assignments/life_expectancy/tests/conftest.py | 9 +++- 4 files changed, 51 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 753fc8c..d50688c 100644 --- a/README.md +++ b/README.md @@ -92,12 +92,12 @@ Week 04 _(~2.5 hours)_ - Assignment #1 -Week 05 _(~2.0 hours)_ +Week 05 _(~2 hours)_ - Git strategies - Assessing code quality: reviews and structures -Week 06 _(~2 hours)_ +Week 06 _(~2.5 hours)_ - Assignment #2 diff --git a/assignments/assignment_3/README.md b/assignments/assignment_3/README.md index 86620bc..f477dd0 100644 --- a/assignments/assignment_3/README.md +++ b/assignments/assignment_3/README.md @@ -2,31 +2,60 @@ Assignment 3 is about testing. -We currently have a single integration test, but that test is very dangerous: it deletes any files created by the application - including ones resulting from its normal usage. Ideally, we would either not have our tests change actual data and not depend on connectivity in order to work, or have a database made specifically for testing. +We currently have a single integration test and this test was very dangerous: before you refactored the clean_data() function, it would overwrite any files created by the application - including ones resulting from its normal usage. Ideally, we would either not have our tests change actual data and not depend on connectivity in order to work, or have a database made specifically for testing. Therefore, let us solve this issue in this exercise. > Don't forget to create a branch for this assignment. +## 0- VSCode Test Discovery + +In order to make your life easier during this assignment, we want to give you a little tip. If you're on VSCode, you can use the [Python Test Explorer](https://marketplace.visualstudio.com/items?itemName=LittleFoxTeam.vscode-python-test-adapter) extension to run your tests. It will automatically discover your tests and you can run them from the Test Explorer tab. At this point, if you have refactored your functions as requested in [assignment_2](../assignment_2/README.md) your tests should be failing. + ## 1- Fixtures -1. Create a fixture based on the `life_expectancy\data\eu_life_expectancy_raw.tsv` data. It can be a sample, so that our tests run faster. Then, replace the `life_expectancy\tests\fixtures\pt_life_expectancy_expected.csv` with the new expected data. -2. Modify your `main` function so that the cleaned DataFrame is always returned. That will allow you to compare it with the expected fixture. -3. Make the tests consume this fixture instead of the actual data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. +0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the european life expectancy dataframe. +1. Following best practises, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume. Since we only want to test our functions, this fixture does not need to be a copy of the original, but could rather be a smaller sample, so that our tests run faster. +2. Now that we have _our_ sample, we need a fixture of the expected output associated to it. You can use your existing `life_expectancy` module to generate a `pt_life_expectancy_expected.csv` file that will replace the current file at `life_expectancy\tests\fixtures\pt_life_expectancy_expected.csv`. Finally, include the necessary code to import this new fixture in the `conftest.py` file. +3. Modify your `main` function so that the cleaned DataFrame is always returned. That will allow you to compare it with the expected fixture. +4. Modify the current test in `test_cleaning.py` to consume these two fixtures instead of the actual data. +> Don't forget to pass the new fixture an argument of the test functions! ## 2- Unit tests and Mocks -1. Right now you should have more python modules. If not, ensure you have at least 2: one for data cleaning and one for loading/saving data. -2. We also have a single integration test, but we should have one unit tests for each of the non-private functions[^1] we have. So, for example, if you have 3 public functions, you should have 3 unit tests. +0. Right now you should have more python modules. If not, ensure you have at least 2: one for data cleaning and one for loading/saving data. +```bash +. +├── LICENSE.md +├── README.md +├── life_expectancy # This directory contains the package you'll be creating +│ ├── __init__.py # This file is required for Python to recognize this directory as a module +│ ├── data # Data files are to be kept in this directory +│ │ └── eu_life_expectancy_raw.tsv +│ ├── temp.py # A temporary file to test the installation. You will delete it later. +│ │── tests # Directory for tests. +│ │ ├── __init__.py +│ │ ├── conftest.py # `conftest.py` is a special pytest file. It contains fixtures and plugins. +│ │ ├── fixtures # Fixtures are reusable objects that can be used in tests. +│ │ │ ├── eu_life_expectancy_expected.csv +│ │ │ └── pt_life_expectancy_expected.csv +│ │ ├── test_cleaning.py # Tests for the cleaning module (assignment 1) +│ │ └── test_pyproject.py # Tests for the pyproject installation (this assignment) +| ├── cleaning +| | ├── __init__.py +| | ├── cleaning.py +| ├── load_save_data +| | ├── __init__.py +| | ├── load_data.py +| | ├── save_data.py +| +└── pyproject.toml +``` +1. We also have a single integration test, but we should have one unit tests for each of the non-private functions[^1] we have. So, for example, if you have 3 public functions, you should have 3 unit tests. 1. On the units above, ensure that any test to functions that save data uses a mock 2. You can do this by patching the `pd.DataFrame.to_csv` method to make the tests not write to a file. Instead, it should just print out a message. 3. Then, assert that the `pd.DataFrame.to_csv` method is being called. As a bonus, by setting up your tests in this fashion, we can ensure no data transformations modifies any actual data. [^1]: Yes, you can also test internal functions if you want to (specially for training purposes). But in the real world, remember that you really believe a particular internal function should be tested, that's a strong indicator it should be decoupled and placed inside its own module - thus making it public. -## 3- VSCode Test Discovery - -If you're on VSCode, you can use the [Python Test Explorer](https://marketplace.visualstudio.com/items?itemName=LittleFoxTeam.vscode-python-test-adapter) extension to run your tests. It will automatically discover your tests and you can run them from the Test Explorer tab. - -1. Run your test suite using the Test Explorer. You should see the integration test and the unit test. ## 4- Code Review diff --git a/assignments/assignment_4/README.md b/assignments/assignment_4/README.md index dae03a1..6f415bb 100644 --- a/assignments/assignment_4/README.md +++ b/assignments/assignment_4/README.md @@ -7,7 +7,7 @@ The focus of this (final) assignment is about OOP and design patterns. And we ha ## 1- Design Patterns 1. Copy the JSON data from the `assignment_4` folder to the `life_expectancy/data` folder. -2. Refactor your code so that is can accept data in other formats if necessary. You can use the [Strategy pattern](https://refactoring.guru/design-patterns/strategy) for this, but you can also use a different one if you prefer. +2. Refactor your code so that it can accept data in other formats if necessary. You can use the [Strategy pattern](https://refactoring.guru/design-patterns/strategy) for this, but you can also use a different one if you prefer. 3. Ensure you still have complete test coverage and a high `pylint` score. ## 2- Enums diff --git a/assignments/life_expectancy/tests/conftest.py b/assignments/life_expectancy/tests/conftest.py index c7c8866..bdfffaf 100644 --- a/assignments/life_expectancy/tests/conftest.py +++ b/assignments/life_expectancy/tests/conftest.py @@ -7,7 +7,14 @@ @pytest.fixture(autouse=True) def run_before_and_after_tests() -> None: - """Fixture to execute commands before and after a test is run""" + """Fixture to execute commands before and after a test is run. + 1. Everything you may write before 'yield' will run before the tests + 2. The command 'yield' marks where tests will happen + 3. The code after 'yield' will be run after the tests are finished. + Thus, before you refactor the clean_data() function into 3 smaller functions, + tests will produce a csv file on the output directory. Since this is bad + practise, we are erasing this file to avoid polluting your workspace. + After refactoring your functions this code will do nothing.""" # Setup: fill with any logic you want yield # this is where the testing happens From 7605f4537c33b3cb6bdd2ddd6ca63fc366cfade8 Mon Sep 17 00:00:00 2001 From: Roberto Vega Date: Thu, 31 Aug 2023 17:44:30 +0100 Subject: [PATCH 03/12] Set Enum exercise as a new assignment, stressing out the importance of VSCode shortcuts. Left the design patterns module and assignment for the last week. --- README.md | 8 ++--- assignments/assignment_4/README.md | 28 ++++++------------ assignments/assignment_5/README.md | 19 ++++++++++++ .../eurostat_life_expect.zip | Bin 4 files changed, 32 insertions(+), 23 deletions(-) create mode 100644 assignments/assignment_5/README.md rename assignments/{assignment_4 => assignment_5}/eurostat_life_expect.zip (100%) diff --git a/README.md b/README.md index d50688c..435880a 100644 --- a/README.md +++ b/README.md @@ -104,23 +104,23 @@ Week 06 _(~2.5 hours)_ Week 07 _(~2 hours)_ - Testing -- Text editors Week 08 _(~2 hours)_ - Assignment #3 Week 09 _(~2.5 hours)_ - +- Text editors - Object-oriented programming Week 10 _(~2.5 hours)_ -- Design patterns +- Assignment #4 Week 11 _(~2 hours)_ -- Assignment #4 +- Design patterns +- Assignment #5 ## Assignments diff --git a/assignments/assignment_4/README.md b/assignments/assignment_4/README.md index 6f415bb..540023e 100644 --- a/assignments/assignment_4/README.md +++ b/assignments/assignment_4/README.md @@ -1,25 +1,15 @@ -# Assignment 4: Design Patterns +# Assignment 4: OOP and VSCode Shortcuts -The focus of this (final) assignment is about OOP and design patterns. And we have bad news: we went to check the Eurostat source and now they are using JSON as a data format. Also, the file was zipped. +The focus of this assignment is about OOP. In particular, we want to make sure that we only provide valid values for our countries. This is the perfect scenario to make use of the Enum class. Note that this is a special case of classes since they are rarely instantiated. -> Don't forget to create a branch for this assignment. - -## 1- Design Patterns - -1. Copy the JSON data from the `assignment_4` folder to the `life_expectancy/data` folder. -2. Refactor your code so that it can accept data in other formats if necessary. You can use the [Strategy pattern](https://refactoring.guru/design-patterns/strategy) for this, but you can also use a different one if you prefer. -3. Ensure you still have complete test coverage and a high `pylint` score. +But going beyond that, our goal is to create the most annoying assignment ever written. The twist is that there is an easy way out: VSCode shortcuts. -## 2- Enums - -Ok, the final stretch. You can do this. Passing a country as a string is not very safe (imprecise types are a code smell). We can use an enum to make sure that we only pass valid countries. - -1. Create an `enum.Enum` called `Country` with possible country values. -2. Then, modify the necessary functions and tests to accept a `Country` instead of a string. Don't forget the type hints. -3. Finally, add a class method to `Country` that returns a list of all the _actual_ countries (so, it removes values like EU28, EFTA, etc). Add a test for this method. +> Don't forget to create a branch for this assignment. -## 3- Code Review +## 1- Enums and Shortcuts -As with the previous assignment, a peer should review your code and you will be reviewing the code of a fellow colleague. +Passing a country as a string is not very safe (imprecise types are a code smell). We can use an enum to make sure that we only pass valid countries. -Was your PR approved? Great! Now you can merge it into `main`, submit the assignment and go celebrate! 🎈 +1. Create an `enum.Enum` called `Country` with possible country values. The idea behind this step is to find the most appropriate VSCode shortcut/s that you learned about in the [text editor](../../07_text_editors/README.md) module. +2. Modify the necessary functions and tests to accept a `Country` instead of a string. Don't forget the type hints. +3. Finally, add a class method to `Country` that returns a list of all the _actual_ countries (so, it removes values like EU28, EFTA, etc). Add a test for this method. Again, make use of as many shortcuts as possible, it will make your life much easier. \ No newline at end of file diff --git a/assignments/assignment_5/README.md b/assignments/assignment_5/README.md new file mode 100644 index 0000000..6438c77 --- /dev/null +++ b/assignments/assignment_5/README.md @@ -0,0 +1,19 @@ +# Assignment 5: Design Patterns + +The focus of this (final) assignment is about design patterns. And we have bad news: we went to check the Eurostat source and now they are using JSON as a data format. Also, the file was zipped. + +> Don't forget to create a branch for this assignment. + +## 1- Design Patterns + +1. Copy the JSON data from the `assignment_5` folder to the `life_expectancy/data` folder. +2. Refactor your code so that it can accept data in other formats if necessary. You can use the [Strategy pattern](https://refactoring.guru/design-patterns/strategy) for this, but you can also use a different one if you prefer. +3. Ensure you still have complete test coverage and a high `pylint` score. + + + +## 2- Code Review + +As with the previous assignment, a peer should review your code and you will be reviewing the code of a fellow colleague. + +Was your PR approved? Great! Now you can merge it into `main`, submit the assignment and go celebrate! 🎈 diff --git a/assignments/assignment_4/eurostat_life_expect.zip b/assignments/assignment_5/eurostat_life_expect.zip similarity index 100% rename from assignments/assignment_4/eurostat_life_expect.zip rename to assignments/assignment_5/eurostat_life_expect.zip From f19746c4e7c0d6e1b08787399b9eda70d1e574ee Mon Sep 17 00:00:00 2001 From: Roberto Vega Date: Thu, 31 Aug 2023 18:08:36 +0100 Subject: [PATCH 04/12] Rewrote the end of some modules to reflect assignment timing --- 06_testing/README.md | 17 +++++++++++++++++ 07_text_editors/README.md | 14 -------------- 08_object_oriented_programming/README.md | 13 +++++++++++++ 09_design_patterns/README.md | 7 ++----- 4 files changed, 32 insertions(+), 19 deletions(-) diff --git a/06_testing/README.md b/06_testing/README.md index 0778e20..60a84c2 100644 --- a/06_testing/README.md +++ b/06_testing/README.md @@ -71,3 +71,20 @@ These are optional, but they can be useful if you want to learn more about testi - [Code coverage](coverage.md) - What is code coverage and how to use it with `pytest`. - [Pytest's `Monkeypatch`](https://docs.pytest.org/en/latest/how-to/monkeypatch.html) - Pytest's version of mock objects is called `Monkeypatch`. They work a little differently and are not as flexible as `unittest.mock`'s mock objects, but they are easier to use. We won't cover them here, but here's a nice - and official - resource on how to use them. + + +## Assignment #3 + +It's finally time for Assignment #3! + +> **Note**: This assignment will require your code to be reviewed by your peers. + +This assignment is about creating tests. It's not the more joyful of tasks - particularly for existing code - but it's oh so important. + +### Assignment #3 covers + +- Creating tests +- Creating test fixtures +- Mocking methods and objects + +`cd` into the `assignments` folder, good luck and see you on the other side! \ No newline at end of file diff --git a/07_text_editors/README.md b/07_text_editors/README.md index a75b906..a42f466 100644 --- a/07_text_editors/README.md +++ b/07_text_editors/README.md @@ -10,18 +10,4 @@ The focus of this module is VSCode (sorry, PyCharm fans!). We cover the basic Py --- -## Assignment #3 -It's finally time for Assignment #3! - -> **Note**: This assignment will require your code to be reviewed by your peers. - -This assignment is about creating tests. It's not the more joyful of tasks - particularly for existing code - but it's oh so important. - -### Assignment #3 covers - -- Creating tests -- Creating test fixtures -- Mocking methods and objects - -`cd` into the `assignments` folder, good luck and see you on the other side! diff --git a/08_object_oriented_programming/README.md b/08_object_oriented_programming/README.md index 5b1bec5..51fd6d2 100644 --- a/08_object_oriented_programming/README.md +++ b/08_object_oriented_programming/README.md @@ -75,3 +75,16 @@ Protocols are similar to abstract data classes. So when should you use a protoco After this lesson you should: - Know what Protocol classes are and when to use them. + + +## Assignment #4 + +This assignment is about OOP and VSCode shortcuts, make sure you have those in mind! + +### Assignment #4 covers + +- Enums +- Object composition +- Testing objects + +`cd` into the `assignments` folder, and let's go! diff --git a/09_design_patterns/README.md b/09_design_patterns/README.md index 90a5351..e0d13cf 100644 --- a/09_design_patterns/README.md +++ b/09_design_patterns/README.md @@ -72,7 +72,7 @@ With the (Abstract) Factory pattern, you can separate creation from usage. It al --- -## Assignment #4 +## Assignment #5 It's the final assignment! Are you ready? If so, chaaaarge! ╰（‵□′）╯ @@ -80,12 +80,9 @@ It's the final assignment! Are you ready? If so, chaaaarge! ╰（‵□′） This assignment is about using design patterns to improve your code maintainability. -### Assignment #4 covers +### Assignment #5 covers -- Enums - Design patterns -- Object composition -- Testing objects `cd` into the `assignments` folder, and let's go! From 8fd920e5aab4fbdbcc6f462fb151cf48ae6c0860 Mon Sep 17 00:00:00 2001 From: RoberVega <109436592+RoberVega@users.noreply.github.com> Date: Fri, 1 Sep 2023 14:31:57 +0100 Subject: [PATCH 05/12] Apply suggestions from code review Co-authored-by: Fernando --- 06_testing/README.md | 2 +- assignments/assignment_3/README.md | 6 ++---- assignments/assignment_4/README.md | 8 ++++---- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/06_testing/README.md b/06_testing/README.md index 60a84c2..d3b87d7 100644 --- a/06_testing/README.md +++ b/06_testing/README.md @@ -87,4 +87,4 @@ This assignment is about creating tests. It's not the more joyful of tasks - par - Creating test fixtures - Mocking methods and objects -`cd` into the `assignments` folder, good luck and see you on the other side! \ No newline at end of file +Open the `assignments` project, good luck and see you on the other side! \ No newline at end of file diff --git a/assignments/assignment_3/README.md b/assignments/assignment_3/README.md index f477dd0..ba88dd4 100644 --- a/assignments/assignment_3/README.md +++ b/assignments/assignment_3/README.md @@ -12,7 +12,7 @@ In order to make your life easier during this assignment, we want to give you a ## 1- Fixtures -0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the european life expectancy dataframe. +0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the expected european life expectancy dataframe (what is, without the PT filter). 1. Following best practises, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume. Since we only want to test our functions, this fixture does not need to be a copy of the original, but could rather be a smaller sample, so that our tests run faster. 2. Now that we have _our_ sample, we need a fixture of the expected output associated to it. You can use your existing `life_expectancy` module to generate a `pt_life_expectancy_expected.csv` file that will replace the current file at `life_expectancy\tests\fixtures\pt_life_expectancy_expected.csv`. Finally, include the necessary code to import this new fixture in the `conftest.py` file. 3. Modify your `main` function so that the cleaned DataFrame is always returned. That will allow you to compare it with the expected fixture. @@ -30,13 +30,11 @@ In order to make your life easier during this assignment, we want to give you a │ ├── __init__.py # This file is required for Python to recognize this directory as a module │ ├── data # Data files are to be kept in this directory │ │ └── eu_life_expectancy_raw.tsv -│ ├── temp.py # A temporary file to test the installation. You will delete it later. │ │── tests # Directory for tests. │ │ ├── __init__.py │ │ ├── conftest.py # `conftest.py` is a special pytest file. It contains fixtures and plugins. │ │ ├── fixtures # Fixtures are reusable objects that can be used in tests. -│ │ │ ├── eu_life_expectancy_expected.csv -│ │ │ └── pt_life_expectancy_expected.csv +│ │ │ └── │ │ ├── test_cleaning.py # Tests for the cleaning module (assignment 1) │ │ └── test_pyproject.py # Tests for the pyproject installation (this assignment) | ├── cleaning diff --git a/assignments/assignment_4/README.md b/assignments/assignment_4/README.md index 540023e..44cf832 100644 --- a/assignments/assignment_4/README.md +++ b/assignments/assignment_4/README.md @@ -1,6 +1,6 @@ # Assignment 4: OOP and VSCode Shortcuts -The focus of this assignment is about OOP. In particular, we want to make sure that we only provide valid values for our countries. This is the perfect scenario to make use of the Enum class. Note that this is a special case of classes since they are rarely instantiated. +The focus of this assignment is about OOP. In particular, we want to make sure that we only provide valid values for our countries. This is the perfect scenario to make use of the Enum class. But going beyond that, our goal is to create the most annoying assignment ever written. The twist is that there is an easy way out: VSCode shortcuts. @@ -10,6 +10,6 @@ But going beyond that, our goal is to create the most annoying assignment ever w Passing a country as a string is not very safe (imprecise types are a code smell). We can use an enum to make sure that we only pass valid countries. -1. Create an `enum.Enum` called `Country` with possible country values. The idea behind this step is to find the most appropriate VSCode shortcut/s that you learned about in the [text editor](../../07_text_editors/README.md) module. -2. Modify the necessary functions and tests to accept a `Country` instead of a string. Don't forget the type hints. -3. Finally, add a class method to `Country` that returns a list of all the _actual_ countries (so, it removes values like EU28, EFTA, etc). Add a test for this method. Again, make use of as many shortcuts as possible, it will make your life much easier. \ No newline at end of file +1. Create an `enum.Enum` called `Region` with **ALL** possible region values. You will have to extract the values from the pandas data frame and then copy the result to your `Region` class. Use the most appropriate VSCode shortcut/s that you learned about in the [text editor](../../07_text_editors/README.md) module to make your job easier! +2. Modify the necessary functions and tests to accept a `Region` instead of a string. Don't forget the type hints. +3. Finally, add a class method to `Region` that returns a list of all the _actual_ countries (so, it removes values like EU28, EFTA, etc). Add a test for this method. Again, make use of as many shortcuts as possible, it will make your life much easier. \ No newline at end of file From e90acc74787195da6996145ad16046d327023c5f Mon Sep 17 00:00:00 2001 From: Roberto Vega Date: Mon, 4 Sep 2023 07:34:52 +0100 Subject: [PATCH 06/12] Implemented the suggested changes. Also, switched the OOP and Text editors modules. --- .../README.md | 0 .../1_setup.md | 0 .../2_productivity.md | 0 .../3_debugger.md | 0 .../README.md | 0 README.md | 5 +++-- assignments/assignment_3/README.md | 19 +++++++++++-------- assignments/assignment_4/README.md | 8 ++++++-- 8 files changed, 20 insertions(+), 12 deletions(-) rename {08_object_oriented_programming => 07_object_oriented_programming}/README.md (100%) rename {07_text_editors => 08_text_editors}/1_setup.md (100%) rename {07_text_editors => 08_text_editors}/2_productivity.md (100%) rename {07_text_editors => 08_text_editors}/3_debugger.md (100%) rename {07_text_editors => 08_text_editors}/README.md (100%) diff --git a/08_object_oriented_programming/README.md b/07_object_oriented_programming/README.md similarity index 100% rename from 08_object_oriented_programming/README.md rename to 07_object_oriented_programming/README.md diff --git a/07_text_editors/1_setup.md b/08_text_editors/1_setup.md similarity index 100% rename from 07_text_editors/1_setup.md rename to 08_text_editors/1_setup.md diff --git a/07_text_editors/2_productivity.md b/08_text_editors/2_productivity.md similarity index 100% rename from 07_text_editors/2_productivity.md rename to 08_text_editors/2_productivity.md diff --git a/07_text_editors/3_debugger.md b/08_text_editors/3_debugger.md similarity index 100% rename from 07_text_editors/3_debugger.md rename to 08_text_editors/3_debugger.md diff --git a/07_text_editors/README.md b/08_text_editors/README.md similarity index 100% rename from 07_text_editors/README.md rename to 08_text_editors/README.md diff --git a/README.md b/README.md index 435880a..1fa4b56 100644 --- a/README.md +++ b/README.md @@ -110,11 +110,12 @@ Week 08 _(~2 hours)_ - Assignment #3 Week 09 _(~2.5 hours)_ -- Text editors + - Object-oriented programming -Week 10 _(~2.5 hours)_ +Week 10 _(~3.5 hours)_ +- Text editors - Assignment #4 Week 11 _(~2 hours)_ diff --git a/assignments/assignment_3/README.md b/assignments/assignment_3/README.md index ba88dd4..d040f33 100644 --- a/assignments/assignment_3/README.md +++ b/assignments/assignment_3/README.md @@ -13,11 +13,17 @@ In order to make your life easier during this assignment, we want to give you a ## 1- Fixtures 0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the expected european life expectancy dataframe (what is, without the PT filter). -1. Following best practises, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume. Since we only want to test our functions, this fixture does not need to be a copy of the original, but could rather be a smaller sample, so that our tests run faster. +1. Following best practises, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume---see substeps below. +> Note: the only reason why you can safely create this fixture is because you previously had a passing test, so before using your functions to create the fixture, you already knew that your functions were doing what they were supposed to. In a real-world scenario without previous tests, you would have first to manually make sure that your procedures and functions are correct, otherwise you would create lying fixtures!! + +Since we only want to test our functions, this fixture does not need to be a copy of the original, but could rather be a smaller sample, so that our tests run faster: + * Starting from `life_expectancy\data\eu_life_expectancy_raw.tsv` create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` + * Since you have changed `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` to be a sample, now your expected output _should_ be the corresponding image of this sample! Therefore, use your current functions to generate the corresponding `life_expectancy\tests\fixtures\eu_life_expectancy_expected.csv` file. 2. Now that we have _our_ sample, we need a fixture of the expected output associated to it. You can use your existing `life_expectancy` module to generate a `pt_life_expectancy_expected.csv` file that will replace the current file at `life_expectancy\tests\fixtures\pt_life_expectancy_expected.csv`. Finally, include the necessary code to import this new fixture in the `conftest.py` file. 3. Modify your `main` function so that the cleaned DataFrame is always returned. That will allow you to compare it with the expected fixture. 4. Modify the current test in `test_cleaning.py` to consume these two fixtures instead of the actual data. > Don't forget to pass the new fixture an argument of the test functions! +5. Final touch: if you read the docstring of the `run_before_and_after_tests` function in the `life_expectancy\tests\conftest.py` file, you will see that by now, the function will be doing nothing... and, if it does nothing, it's dead code and dead code must be removed :smiling_imp:! ## 2- Unit tests and Mocks @@ -37,13 +43,10 @@ In order to make your life easier during this assignment, we want to give you a │ │ │ └── │ │ ├── test_cleaning.py # Tests for the cleaning module (assignment 1) │ │ └── test_pyproject.py # Tests for the pyproject installation (this assignment) -| ├── cleaning -| | ├── __init__.py -| | ├── cleaning.py -| ├── load_save_data -| | ├── __init__.py -| | ├── load_data.py -| | ├── save_data.py +| ├── module_1.py # Your modules +| ├── module_2.py +| ├── ... +| └── module_n.py | └── pyproject.toml ``` diff --git a/assignments/assignment_4/README.md b/assignments/assignment_4/README.md index 44cf832..ece3dfd 100644 --- a/assignments/assignment_4/README.md +++ b/assignments/assignment_4/README.md @@ -1,4 +1,4 @@ -# Assignment 4: OOP and VSCode Shortcuts +# Assignment 4: Enums and Text Editors The focus of this assignment is about OOP. In particular, we want to make sure that we only provide valid values for our countries. This is the perfect scenario to make use of the Enum class. @@ -12,4 +12,8 @@ Passing a country as a string is not very safe (imprecise types are a code smell 1. Create an `enum.Enum` called `Region` with **ALL** possible region values. You will have to extract the values from the pandas data frame and then copy the result to your `Region` class. Use the most appropriate VSCode shortcut/s that you learned about in the [text editor](../../07_text_editors/README.md) module to make your job easier! 2. Modify the necessary functions and tests to accept a `Region` instead of a string. Don't forget the type hints. -3. Finally, add a class method to `Region` that returns a list of all the _actual_ countries (so, it removes values like EU28, EFTA, etc). Add a test for this method. Again, make use of as many shortcuts as possible, it will make your life much easier. \ No newline at end of file +3. Finally, add a class method to `Region` that returns a list of all the _actual_ countries (so, it removes values like EU28, EFTA, etc). Add a test for this method. Again, make use of as many shortcuts as possible, it will make your life much easier. + +## 4- Code Review + +As with the previous assignment, a peer should review your code and you will be reviewing the code of a fellow colleague. From e24d66c164c74858bb5fea199b39907dcca4d8d7 Mon Sep 17 00:00:00 2001 From: Fernando Cordeiro Date: Mon, 4 Sep 2023 10:28:27 +0100 Subject: [PATCH 07/12] =?UTF-8?q?=F0=9F=91=95=20LINT:=20Fix=20markdownlint?= =?UTF-8?q?=20violations?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- 06_testing/README.md | 3 +- 07_object_oriented_programming/README.md | 2 +- assignments/assignment_3/README.md | 62 +++++++++++++----------- assignments/assignment_4/README.md | 4 +- assignments/assignment_5/README.md | 2 - 5 files changed, 38 insertions(+), 35 deletions(-) diff --git a/06_testing/README.md b/06_testing/README.md index d3b87d7..7b5d2d6 100644 --- a/06_testing/README.md +++ b/06_testing/README.md @@ -72,7 +72,6 @@ These are optional, but they can be useful if you want to learn more about testi - [Code coverage](coverage.md) - What is code coverage and how to use it with `pytest`. - [Pytest's `Monkeypatch`](https://docs.pytest.org/en/latest/how-to/monkeypatch.html) - Pytest's version of mock objects is called `Monkeypatch`. They work a little differently and are not as flexible as `unittest.mock`'s mock objects, but they are easier to use. We won't cover them here, but here's a nice - and official - resource on how to use them. - ## Assignment #3 It's finally time for Assignment #3! @@ -87,4 +86,4 @@ This assignment is about creating tests. It's not the more joyful of tasks - par - Creating test fixtures - Mocking methods and objects -Open the `assignments` project, good luck and see you on the other side! \ No newline at end of file +Open the `assignments` project, good luck and see you on the other side! diff --git a/07_object_oriented_programming/README.md b/07_object_oriented_programming/README.md index 51fd6d2..d342aaa 100644 --- a/07_object_oriented_programming/README.md +++ b/07_object_oriented_programming/README.md @@ -76,7 +76,6 @@ After this lesson you should: - Know what Protocol classes are and when to use them. - ## Assignment #4 This assignment is about OOP and VSCode shortcuts, make sure you have those in mind! @@ -88,3 +87,4 @@ This assignment is about OOP and VSCode shortcuts, make sure you have those in m - Testing objects `cd` into the `assignments` folder, and let's go! + diff --git a/assignments/assignment_3/README.md b/assignments/assignment_3/README.md index d040f33..6a99976 100644 --- a/assignments/assignment_3/README.md +++ b/assignments/assignment_3/README.md @@ -13,43 +13,50 @@ In order to make your life easier during this assignment, we want to give you a ## 1- Fixtures 0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the expected european life expectancy dataframe (what is, without the PT filter). -1. Following best practises, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume---see substeps below. -> Note: the only reason why you can safely create this fixture is because you previously had a passing test, so before using your functions to create the fixture, you already knew that your functions were doing what they were supposed to. In a real-world scenario without previous tests, you would have first to manually make sure that your procedures and functions are correct, otherwise you would create lying fixtures!! +1. Following best practises, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume---see substeps below. + + > Note: the only reason why you can safely create this fixture is because you previously had a passing test, so before using your functions to create the fixture, you already knew that your functions were doing what they were supposed to. In a real-world scenario without previous tests, you would have first to manually make sure that your procedures and functions are correct, otherwise you would create lying fixtures!! + + Since we only want to test our functions, this fixture does not need to be a copy of the original, but could rather be a smaller sample, so that our tests run faster: -Since we only want to test our functions, this fixture does not need to be a copy of the original, but could rather be a smaller sample, so that our tests run faster: * Starting from `life_expectancy\data\eu_life_expectancy_raw.tsv` create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` * Since you have changed `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` to be a sample, now your expected output _should_ be the corresponding image of this sample! Therefore, use your current functions to generate the corresponding `life_expectancy\tests\fixtures\eu_life_expectancy_expected.csv` file. + 2. Now that we have _our_ sample, we need a fixture of the expected output associated to it. You can use your existing `life_expectancy` module to generate a `pt_life_expectancy_expected.csv` file that will replace the current file at `life_expectancy\tests\fixtures\pt_life_expectancy_expected.csv`. Finally, include the necessary code to import this new fixture in the `conftest.py` file. 3. Modify your `main` function so that the cleaned DataFrame is always returned. That will allow you to compare it with the expected fixture. 4. Modify the current test in `test_cleaning.py` to consume these two fixtures instead of the actual data. -> Don't forget to pass the new fixture an argument of the test functions! + + > Don't forget to pass the new fixture an argument of the test functions! + 5. Final touch: if you read the docstring of the `run_before_and_after_tests` function in the `life_expectancy\tests\conftest.py` file, you will see that by now, the function will be doing nothing... and, if it does nothing, it's dead code and dead code must be removed :smiling_imp:! ## 2- Unit tests and Mocks -0. Right now you should have more python modules. If not, ensure you have at least 2: one for data cleaning and one for loading/saving data. -```bash -. -├── LICENSE.md -├── README.md -├── life_expectancy # This directory contains the package you'll be creating -│ ├── __init__.py # This file is required for Python to recognize this directory as a module -│ ├── data # Data files are to be kept in this directory -│ │ └── eu_life_expectancy_raw.tsv -│ │── tests # Directory for tests. -│ │ ├── __init__.py -│ │ ├── conftest.py # `conftest.py` is a special pytest file. It contains fixtures and plugins. -│ │ ├── fixtures # Fixtures are reusable objects that can be used in tests. -│ │ │ └── -│ │ ├── test_cleaning.py # Tests for the cleaning module (assignment 1) -│ │ └── test_pyproject.py # Tests for the pyproject installation (this assignment) -| ├── module_1.py # Your modules -| ├── module_2.py -| ├── ... -| └── module_n.py -| -└── pyproject.toml -``` +0. Right now you should have more python modules. If not, ensure you have at least 2: one for data cleaning and one for loading/saving data. + + ```bash + . + ├── LICENSE.md + ├── README.md + ├── life_expectancy # This directory contains the package you'll be creating + │ ├── __init__.py # This file is required for Python to recognize this directory as a module + │ ├── data # Data files are to be kept in this directory + │ │ └── eu_life_expectancy_raw.tsv + │ │── tests # Directory for tests. + │ │ ├── __init__.py + │ │ ├── conftest.py # `conftest.py` is a special pytest file. It contains fixtures and plugins. + │ │ ├── fixtures # Fixtures are reusable objects that can be used in tests. + │ │ │ └── + │ │ ├── test_cleaning.py # Tests for the cleaning module (assignment 1) + │ │ └── test_pyproject.py # Tests for the pyproject installation (this assignment) + | ├── module_1.py # Your modules + | ├── module_2.py + | ├── ... + | └── module_n.py + | + └── pyproject.toml + ``` + 1. We also have a single integration test, but we should have one unit tests for each of the non-private functions[^1] we have. So, for example, if you have 3 public functions, you should have 3 unit tests. 1. On the units above, ensure that any test to functions that save data uses a mock 2. You can do this by patching the `pd.DataFrame.to_csv` method to make the tests not write to a file. Instead, it should just print out a message. @@ -57,7 +64,6 @@ Since we only want to test our functions, this fixture does not need to be a cop [^1]: Yes, you can also test internal functions if you want to (specially for training purposes). But in the real world, remember that you really believe a particular internal function should be tested, that's a strong indicator it should be decoupled and placed inside its own module - thus making it public. - ## 4- Code Review As with the previous assignment, a peer should review your code and you will be reviewing the code of a fellow colleague. diff --git a/assignments/assignment_4/README.md b/assignments/assignment_4/README.md index ece3dfd..61203f5 100644 --- a/assignments/assignment_4/README.md +++ b/assignments/assignment_4/README.md @@ -1,6 +1,6 @@ # Assignment 4: Enums and Text Editors -The focus of this assignment is about OOP. In particular, we want to make sure that we only provide valid values for our countries. This is the perfect scenario to make use of the Enum class. +The focus of this assignment is about OOP. In particular, we want to make sure that we only provide valid values for our countries. This is the perfect scenario to make use of the Enum class. But going beyond that, our goal is to create the most annoying assignment ever written. The twist is that there is an easy way out: VSCode shortcuts. @@ -10,7 +10,7 @@ But going beyond that, our goal is to create the most annoying assignment ever w Passing a country as a string is not very safe (imprecise types are a code smell). We can use an enum to make sure that we only pass valid countries. -1. Create an `enum.Enum` called `Region` with **ALL** possible region values. You will have to extract the values from the pandas data frame and then copy the result to your `Region` class. Use the most appropriate VSCode shortcut/s that you learned about in the [text editor](../../07_text_editors/README.md) module to make your job easier! +1. Create an `enum.Enum` called `Region` with **ALL** possible region values. You will have to extract the values from the pandas data frame and then copy the result to your `Region` class. Use the most appropriate VSCode shortcut/s that you learned about in the [text editor](../../07_text_editors/README.md) module to make your job easier! 2. Modify the necessary functions and tests to accept a `Region` instead of a string. Don't forget the type hints. 3. Finally, add a class method to `Region` that returns a list of all the _actual_ countries (so, it removes values like EU28, EFTA, etc). Add a test for this method. Again, make use of as many shortcuts as possible, it will make your life much easier. diff --git a/assignments/assignment_5/README.md b/assignments/assignment_5/README.md index 6438c77..c7d7b60 100644 --- a/assignments/assignment_5/README.md +++ b/assignments/assignment_5/README.md @@ -10,8 +10,6 @@ The focus of this (final) assignment is about design patterns. And we have bad n 2. Refactor your code so that it can accept data in other formats if necessary. You can use the [Strategy pattern](https://refactoring.guru/design-patterns/strategy) for this, but you can also use a different one if you prefer. 3. Ensure you still have complete test coverage and a high `pylint` score. - - ## 2- Code Review As with the previous assignment, a peer should review your code and you will be reviewing the code of a fellow colleague. From ab0291605a25826e2c4c7c4979a4885d88753d91 Mon Sep 17 00:00:00 2001 From: Fernando Cordeiro Date: Mon, 4 Sep 2023 10:29:15 +0100 Subject: [PATCH 08/12] =?UTF-8?q?=F0=9F=90=9B=20FIX:=20Typos?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- assignments/assignment_3/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/assignments/assignment_3/README.md b/assignments/assignment_3/README.md index 6a99976..8164938 100644 --- a/assignments/assignment_3/README.md +++ b/assignments/assignment_3/README.md @@ -13,7 +13,7 @@ In order to make your life easier during this assignment, we want to give you a ## 1- Fixtures 0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the expected european life expectancy dataframe (what is, without the PT filter). -1. Following best practises, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume---see substeps below. +1. Following best practices, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume - see sub steps below. > Note: the only reason why you can safely create this fixture is because you previously had a passing test, so before using your functions to create the fixture, you already knew that your functions were doing what they were supposed to. In a real-world scenario without previous tests, you would have first to manually make sure that your procedures and functions are correct, otherwise you would create lying fixtures!! From 3a2dfca1887e5c72af549553448fb947bb7d389c Mon Sep 17 00:00:00 2001 From: Fernando Cordeiro Date: Mon, 4 Sep 2023 10:35:43 +0100 Subject: [PATCH 09/12] =?UTF-8?q?=F0=9F=91=8C=20IMPROVE:=20Move=20note=20t?= =?UTF-8?q?o=20footnote?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- assignments/assignment_3/README.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/assignments/assignment_3/README.md b/assignments/assignment_3/README.md index 8164938..964d0a0 100644 --- a/assignments/assignment_3/README.md +++ b/assignments/assignment_3/README.md @@ -13,13 +13,9 @@ In order to make your life easier during this assignment, we want to give you a ## 1- Fixtures 0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the expected european life expectancy dataframe (what is, without the PT filter). -1. Following best practices, we want to create a fixture that represents our data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume - see sub steps below. +1. We want to create a fixture that represents our input data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume - see sub steps below. - > Note: the only reason why you can safely create this fixture is because you previously had a passing test, so before using your functions to create the fixture, you already knew that your functions were doing what they were supposed to. In a real-world scenario without previous tests, you would have first to manually make sure that your procedures and functions are correct, otherwise you would create lying fixtures!! - - Since we only want to test our functions, this fixture does not need to be a copy of the original, but could rather be a smaller sample, so that our tests run faster: - - * Starting from `life_expectancy\data\eu_life_expectancy_raw.tsv` create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` + * Starting from `life_expectancy\data\eu_life_expectancy_raw.tsv` create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` [^1] * Since you have changed `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` to be a sample, now your expected output _should_ be the corresponding image of this sample! Therefore, use your current functions to generate the corresponding `life_expectancy\tests\fixtures\eu_life_expectancy_expected.csv` file. 2. Now that we have _our_ sample, we need a fixture of the expected output associated to it. You can use your existing `life_expectancy` module to generate a `pt_life_expectancy_expected.csv` file that will replace the current file at `life_expectancy\tests\fixtures\pt_life_expectancy_expected.csv`. Finally, include the necessary code to import this new fixture in the `conftest.py` file. @@ -30,6 +26,8 @@ In order to make your life easier during this assignment, we want to give you a 5. Final touch: if you read the docstring of the `run_before_and_after_tests` function in the `life_expectancy\tests\conftest.py` file, you will see that by now, the function will be doing nothing... and, if it does nothing, it's dead code and dead code must be removed :smiling_imp:! +[^1]: The only reason why you can safely create this fixture is because we had that passing test on the 1st assignment. That's how you knew whether your code was doing what they it supposed to. In a real-world scenario without previous tests, you would first have to manually make sure that your procedures and functions are correct, otherwise you would create lying fixtures (i.e. fixtures that assume your code is correct when, in fact, it is not)! + ## 2- Unit tests and Mocks 0. Right now you should have more python modules. If not, ensure you have at least 2: one for data cleaning and one for loading/saving data. From 0ebda010585ad86c3a1ce3fbd33c07a0e6a4b209 Mon Sep 17 00:00:00 2001 From: Fernando Cordeiro Date: Mon, 4 Sep 2023 10:47:05 +0100 Subject: [PATCH 10/12] =?UTF-8?q?=F0=9F=91=8C=20IMPROVE:=20Text=20revision?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- assignments/assignment_3/README.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/assignments/assignment_3/README.md b/assignments/assignment_3/README.md index 964d0a0..6355a97 100644 --- a/assignments/assignment_3/README.md +++ b/assignments/assignment_3/README.md @@ -13,12 +13,12 @@ In order to make your life easier during this assignment, we want to give you a ## 1- Fixtures 0. If you recall the structure from [assignment_0](../assignment_0/README.md), we currently have two fixtures, one for the expected output of the portuguese life expectancy dataframe and another one for the expected european life expectancy dataframe (what is, without the PT filter). -1. We want to create a fixture that represents our input data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Start from the current data in `life_expectancy\data\eu_life_expectancy_raw.tsv` and create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` for the tests to consume - see sub steps below. +1. We want to create a fixture that represents our input data. The idea of using fixtures instead of collecting data from a database will ensure our code is not dependent on connectivity. This will make our lives easier when we want to run our tests in a CI/CD pipeline. Here's how to create them: - * Starting from `life_expectancy\data\eu_life_expectancy_raw.tsv` create a sample fixture `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` [^1] - * Since you have changed `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` to be a sample, now your expected output _should_ be the corresponding image of this sample! Therefore, use your current functions to generate the corresponding `life_expectancy\tests\fixtures\eu_life_expectancy_expected.csv` file. + * **Input fixture**: Let's create our new input fixture by loading the `life_expectancy\data\eu_life_expectancy_raw.tsv`, creating a sample, and saving it as `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` [^1]. Remember that your sample must contain the elements you wish to test. For example, if you want to filter a given region, you fixture has to contain at least one row with that region. + * **Expected fixture**: Now let's create the fixture for the expected result. Since you have changed `life_expectancy\tests\fixtures\eu_life_expectancy_raw.tsv` to be a sample, now your expected output _should_ be the corresponding image of this sample! Therefore, use your code on that input fixture data to generate the corresponding `life_expectancy\tests\fixtures\eu_life_expectancy_expected.csv` file. -2. Now that we have _our_ sample, we need a fixture of the expected output associated to it. You can use your existing `life_expectancy` module to generate a `pt_life_expectancy_expected.csv` file that will replace the current file at `life_expectancy\tests\fixtures\pt_life_expectancy_expected.csv`. Finally, include the necessary code to import this new fixture in the `conftest.py` file. +2. Finally, include the necessary code to import these new fixtures in the `conftest.py` file. 3. Modify your `main` function so that the cleaned DataFrame is always returned. That will allow you to compare it with the expected fixture. 4. Modify the current test in `test_cleaning.py` to consume these two fixtures instead of the actual data. @@ -30,7 +30,7 @@ In order to make your life easier during this assignment, we want to give you a ## 2- Unit tests and Mocks -0. Right now you should have more python modules. If not, ensure you have at least 2: one for data cleaning and one for loading/saving data. +0. Right now you should have more python modules. If not, ensure you have at least 2: one for data cleaning and one for loading/saving data [^2]. ```bash . @@ -55,12 +55,13 @@ In order to make your life easier during this assignment, we want to give you a └── pyproject.toml ``` -1. We also have a single integration test, but we should have one unit tests for each of the non-private functions[^1] we have. So, for example, if you have 3 public functions, you should have 3 unit tests. +1. We also have a single integration test, but we should have one unit tests for each of the non-private functions[^3] we have. So, for example, if you have 3 public functions, you should have 3 unit tests. 1. On the units above, ensure that any test to functions that save data uses a mock 2. You can do this by patching the `pd.DataFrame.to_csv` method to make the tests not write to a file. Instead, it should just print out a message. 3. Then, assert that the `pd.DataFrame.to_csv` method is being called. As a bonus, by setting up your tests in this fashion, we can ensure no data transformations modifies any actual data. -[^1]: Yes, you can also test internal functions if you want to (specially for training purposes). But in the real world, remember that you really believe a particular internal function should be tested, that's a strong indicator it should be decoupled and placed inside its own module - thus making it public. +[^2]: A module is just a python script (or a folder with a `__init__.py` file) that's meant to be imported instead of run directly. +[^3]: Yes, you can also test internal functions if you want to (specially for training purposes). But in the real world, remember that you really believe a particular internal function should be tested, that's a strong indicator it should be decoupled and placed inside its own module - thus making it public. ## 4- Code Review From 54687605a0d970f159777b114c834e93e2e67b67 Mon Sep 17 00:00:00 2001 From: Roberto Vega Date: Mon, 11 Sep 2023 11:02:17 +0100 Subject: [PATCH 11/12] Renumbered sections + 06_debugger + debugging exercise in assignment_2 --- 08_text_editors/3_debugger.md => 06_debugger/README.md | 0 {06_testing => 07_testing}/README.md | 0 {06_testing => 07_testing}/coverage.md | 0 {06_testing => 07_testing}/testing_dataframes.md | 0 .../README.md | 0 {08_text_editors => 09_text_editors}/1_setup.md | 0 {08_text_editors => 09_text_editors}/2_productivity.md | 0 {08_text_editors => 09_text_editors}/README.md | 0 {09_design_patterns => 10_design_patterns}/README.md | 0 README.md | 5 +++-- assignments/assignment_2/README.md | 9 ++++++++- 11 files changed, 11 insertions(+), 3 deletions(-) rename 08_text_editors/3_debugger.md => 06_debugger/README.md (100%) rename {06_testing => 07_testing}/README.md (100%) rename {06_testing => 07_testing}/coverage.md (100%) rename {06_testing => 07_testing}/testing_dataframes.md (100%) rename {07_object_oriented_programming => 08_object_oriented_programming}/README.md (100%) rename {08_text_editors => 09_text_editors}/1_setup.md (100%) rename {08_text_editors => 09_text_editors}/2_productivity.md (100%) rename {08_text_editors => 09_text_editors}/README.md (100%) rename {09_design_patterns => 10_design_patterns}/README.md (100%) diff --git a/08_text_editors/3_debugger.md b/06_debugger/README.md similarity index 100% rename from 08_text_editors/3_debugger.md rename to 06_debugger/README.md diff --git a/06_testing/README.md b/07_testing/README.md similarity index 100% rename from 06_testing/README.md rename to 07_testing/README.md diff --git a/06_testing/coverage.md b/07_testing/coverage.md similarity index 100% rename from 06_testing/coverage.md rename to 07_testing/coverage.md diff --git a/06_testing/testing_dataframes.md b/07_testing/testing_dataframes.md similarity index 100% rename from 06_testing/testing_dataframes.md rename to 07_testing/testing_dataframes.md diff --git a/07_object_oriented_programming/README.md b/08_object_oriented_programming/README.md similarity index 100% rename from 07_object_oriented_programming/README.md rename to 08_object_oriented_programming/README.md diff --git a/08_text_editors/1_setup.md b/09_text_editors/1_setup.md similarity index 100% rename from 08_text_editors/1_setup.md rename to 09_text_editors/1_setup.md diff --git a/08_text_editors/2_productivity.md b/09_text_editors/2_productivity.md similarity index 100% rename from 08_text_editors/2_productivity.md rename to 09_text_editors/2_productivity.md diff --git a/08_text_editors/README.md b/09_text_editors/README.md similarity index 100% rename from 08_text_editors/README.md rename to 09_text_editors/README.md diff --git a/09_design_patterns/README.md b/10_design_patterns/README.md similarity index 100% rename from 09_design_patterns/README.md rename to 10_design_patterns/README.md diff --git a/README.md b/README.md index 1fa4b56..523d69b 100644 --- a/README.md +++ b/README.md @@ -92,10 +92,11 @@ Week 04 _(~2.5 hours)_ - Assignment #1 -Week 05 _(~2 hours)_ +Week 05 _(~2.5 hours)_ - Git strategies - Assessing code quality: reviews and structures +- Debugging Week 06 _(~2.5 hours)_ @@ -113,7 +114,7 @@ Week 09 _(~2.5 hours)_ - Object-oriented programming -Week 10 _(~3.5 hours)_ +Week 10 _(~3.0 hours)_ - Text editors - Assignment #4 diff --git a/assignments/assignment_2/README.md b/assignments/assignment_2/README.md index d4b95b4..247ddcd 100644 --- a/assignments/assignment_2/README.md +++ b/assignments/assignment_2/README.md @@ -59,7 +59,7 @@ We know that the things presented in the previous modules are orientative guidel This exercise is very simple, we just want you to fish out some current or older project that you are or were part of and review its organization. What would you change? What would you keep? Why? -## 2- Structure recognition +## 2- Structure analysis In collecting a couple of examples for the project structure module we made a mess and lost track of the structure each of the projects. Give it a go and see if you can figure out which patterns do these projects follow. @@ -70,3 +70,10 @@ In collecting a couple of examples for the project structure module we made a me [Link to mysterious project 3](https://github.com/RoberVega/mysterious_project_3) [Link to mysterious project 4](https://github.com/cookiecutter/cookiecutter/tree/main) + + +# Assignment 2: code exploration using a debugger + +The last part of this assignment is easy, but very important. One of the most common scenarios is to stumble upon code you have not written yourself. Especially when facing big projects, this might come across as a daunting task. Fortunately, debuggers come to the rescue! One of the possible use cases for debuggers is understanding a new piece of code, especially for studying its intricacies and connections. By introducing strategic breakpoints, one can understand the intended flow of the project and the different parts it is composed of. + +That situation takes us to our next exercise: in the next module, we will focus on testing. But first, we would like you to use a debugger to try to understand how the current testing code actually works, what is being called, etc. \ No newline at end of file From dcd5112e08a2c9d4a2820e25dca7b41beadb0204 Mon Sep 17 00:00:00 2001 From: Roberto Vega Date: Tue, 19 Sep 2023 17:30:13 +0100 Subject: [PATCH 12/12] Extracted the debugging file from text_editors into its own module and added a debugging sub-exercise on assignment_2 --- .../README.md | 0 08_text_editors/1_setup.md | 97 ------------------- 08_text_editors/2_productivity.md | 27 ------ 08_text_editors/3_debugger.md | 55 ----------- 08_text_editors/README.md | 13 --- assignments/assignment_2/README.md | 25 ++--- 6 files changed, 6 insertions(+), 211 deletions(-) rename {07_object_oriented_programming => 08_object_oriented_programming}/README.md (100%) delete mode 100644 08_text_editors/1_setup.md delete mode 100644 08_text_editors/2_productivity.md delete mode 100644 08_text_editors/3_debugger.md delete mode 100644 08_text_editors/README.md diff --git a/07_object_oriented_programming/README.md b/08_object_oriented_programming/README.md similarity index 100% rename from 07_object_oriented_programming/README.md rename to 08_object_oriented_programming/README.md diff --git a/08_text_editors/1_setup.md b/08_text_editors/1_setup.md deleted file mode 100644 index f1c4656..0000000 --- a/08_text_editors/1_setup.md +++ /dev/null @@ -1,97 +0,0 @@ -# Get started with Python in VS Code (0h20) - -The first thing you need after installing VS Code is to install the [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python). This extension is the official Python plugin for VS Code and it provides a lot of useful features. - -The second thing you need to do is to select a Python interpreter. 9 out of 10 times your get a `ModuleNotFound` error is because you are using the wrong interpreter. The Python extension will try to detect the interpreter for you, but you can also select it manually. - -Make sure you read about deal with environments and the Python interpreter in VS Code: [VSCode docs: Environments](https://code.visualstudio.com/docs/python/environments) - -## Setting up VS Code for Python development (0h10) - -### Configuration - -VSCode is highly customizable. This customization lives in the `.vscode/settings.json` file and can be altered either in that file directly or via the user interface. - -> Make sure you have added the `.vscode/` directory to the `.gitignore` file, so your settings don't end up polluting the Git repository by mistake. - -Here are some useful settings: - ---- - -```json -"editor.rulers": [88, 120] -``` - -Render vertical rulers after a certain number of monospace characters. Use multiple values for multiple rulers. Useful in order to ensure no line is too long. - ---- - -```json -"python.testing.pytestEnabled": true OR false, -"python.testing.unittestEnabled": true OR false, -``` - -Helps VSCode identify the Python testing framework being used: pytest vs. unittest. - ---- - -```json -"python.analysis.typeCheckingMode": strict -``` - -Defines the default rule set for type checking. Options are: - -- `off` - no type checking, -- `basic` - More lenient checker. Useful for when working with some external libraries (like Pandas, for instance), -- `strict` - Shows you any error that happens with typing. - ---- - -```json -"editor.formatOnSave": true -``` - -Formats files when they are saved. This allows your code to follow a consistent style automatically. In order to to this, you will need a formatter, like [black](https://github.com/psf/black). There's usually a different formatter for each code style. If you need more information about code styles, it's useful to read a few examples. For example, [Google's Python styleguide](https://google.github.io/styleguide/pyguide.html) is publicly available. The next setting will allow you to choose a formatter. - ---- - -```json -"python.formatting.provided": "black", -``` - -Option to determine how VSCode should format your code. Possible formatters are `Black`, `Autopep8`, and `Yapf`. - ---- - -```json -"editor.formatOnSaveMode": "file" -``` - -This controls how the file will be formatted. Options are: - -- `file` - formats entire file, -- `modifications` - formats only what was modified (requires version control), -- `modificationsIfAvailable` - attempts to format only what was modified, but if version control is not available, formats the entire file. - ---- - -```json -"editor.codeActionsOnSave": ["source.organizeImports"] -``` - -VSCode allows for other actions to be performed when the file is saved. Each action should be added to the list and will be executed in order. The most popular action is `source.organizeImports` which sorts the file imports. - ---- - -```json -"python.linting.pylintEnabled": true, -"python.linting.enabled": true, -``` - -`python.linting` is where the linting options live. The Python extension supports `Pylint`, `bandit`, `pylama`, `Pydocstyle`, `Pycodestyle`, `Prospector`, `Mypy`, and `Flake8`. - -## Recommended plugins, Tasks and the Test Explorer - -[Link to document](https://docs.google.com/document/u/1/d/1xHJ9Kq9OVsWh4OH8DYB_7dsKW2tzCPZ8FOI9VrGU2SU) - -This document helps you setup VS Code's Test Explorer, Tasks. It also provides some plugins recommendations. diff --git a/08_text_editors/2_productivity.md b/08_text_editors/2_productivity.md deleted file mode 100644 index 80bcf84..0000000 --- a/08_text_editors/2_productivity.md +++ /dev/null @@ -1,27 +0,0 @@ -# Python Productivity on VS Code (0h08) - -Getting into the habit of using shortcuts is the secret to becoming a more productive developer. Multi-curser editing, moving lines up and down, and commenting code are just a few of the shortcuts that can save you a lot of time. - -> Did you know you can also use VSCode to refactor your code? Next time you press the right mouse button, try to select `Refactor` and see what happens! - -![8 Tips to SKYROCKET your Python Productivity on VS Code](../images/10b7c07302d9be872cb991d9c1784df7bf45bbb770db80925415605cf49129b4.png) - -[Link to video](https://youtu.be/slHzJh6pGo8) - -This video shows you tips and shortcuts that can really make your life easier when using VS Code. - -> **One shortcut to rule them all**: If you want to search for any shortcuts on VSCode, use `Ctrl + K` followed by `Ctrl + S` to open the keyboard shortcuts menu. - -Note that, depending on your keyboard, the shortcuts may differ. Here are some options we found: - -- Comment toggle: `Crtl+ ^` or `Ctrl + ~` (instead of `Crtl + /`) - -## BONUS CONTENT - More shortcuts! (0h17) - -You sure learned a bunch of tricks by now. Want to learn a few more? - -[![30 VSCode Keyboard Shortcuts You NEED to Know](../images/7e14212a10f18175238ee3e16b54f85b54126843a0d2c21fee8fdce149257282.png)](https://youtu.be/dI34jrEtmB0) - -[In this video](https://youtu.be/dI34jrEtmB0), once again, our friend Arjan will show you a bunch of useful VS Code keyboard shortcuts that will help use it like a pro! - -After these series of videos, VS Code should have no more secrets for you. \ No newline at end of file diff --git a/08_text_editors/3_debugger.md b/08_text_editors/3_debugger.md deleted file mode 100644 index 411c873..0000000 --- a/08_text_editors/3_debugger.md +++ /dev/null @@ -1,55 +0,0 @@ -# The VSCode Debugger (0h30) - -## How to use the debugger (0h17) - -![How to Use a Debugger - Debugger Tutorial](../images/how-to-use-a-debugger.png) - -[Link to video](https://youtu.be/7qZBwhSlfOo?si=D19j0n9lwvITiEN2) - -This video is a great introduction to the concept of debugging. It explains what a debugger is and how it can help you find bugs in your code. - -## Advanced debugging (0h07) - -![Three Python debugger features that will help you fix bugs faster in VS Code](../images/three-python-debugger-features.png) - -[Link to video](https://youtu.be/2hZ5xwgosso?si=Zj_GhbAKGBaBYgal) - -This video shows you how to use some of the more advanced features of the VS Code debugger. Did you know you can change the value of a variable while debugging? Or that you can set breakpoints on specific conditions? This video will show you how to do that. - -## Adding additional configurations (0h05) - -The videos above all run the debugger on specific files. However, what if the thing you want to debug is a function in a file that is imported by another file? Or if you want to debug the testing suite? In these cases, you will need to add additional configurations. - -By clicking on the debug icon on the left, you will open the debug panel. At the top of the panel, you will see a dropdown menu that says `Python File`. Click on it and select `Add Configuration...`. This will open a `launch.json` file. This file contains all the configurations for the debugger. You can add new configurations by adding new objects to the `configurations` array. Here is an example of a configuration that runs the debugger on a specific file `my_file.py`: - -```json -{ - "name": "Python: my_file.py", # The name that will show on the menu - "type": "python", - "request": "launch", - "program": "${workspaceFolder}/my_file.py", - "console": "integratedTerminal" -} -``` - -Read [how to set configuration options](https://code.visualstudio.com/docs/python/debugging#_set-configuration-options) to learn more about the different options you can set. - -One notable configuration is the `python` option. If VS Code doesn't automatically select the right Python environment for you, then you can also declare an explicitly using it. For example: - -```json -{ - "configurations": [ - { - "name": "Python: Streamlit", - "type": "python", - "python": "${workspaceFolder}/venv/bin/python", # Set this to the path of your Python interpreter - "request": "launch", - "module": "streamlit", - "cwd": "${workspaceFolder}", - "args": [ - "run", "app.py" - ], - } - ] -} -``` diff --git a/08_text_editors/README.md b/08_text_editors/README.md deleted file mode 100644 index a42f466..0000000 --- a/08_text_editors/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# Text editors - -The focus of this module is VSCode (sorry, PyCharm fans!). We cover the basic Python setup, productivity tips, and how to use VSCode to its full potential. - -## Call to Action - -1. Follow the [Setup tutorial for VSCode](./1_setup.md) and get your VSCode ready for Python development. -2. Master [Productivity on VSCode](./2_productivity.md) and learn how to use shortcuts to become a more productive developer. -3. Learn how to [Debug on VSCode](./3_debugging.md). Your `print` statements days are over! - ---- - - diff --git a/assignments/assignment_2/README.md b/assignments/assignment_2/README.md index 34fa6ca..71808b4 100644 --- a/assignments/assignment_2/README.md +++ b/assignments/assignment_2/README.md @@ -61,24 +61,6 @@ This exercise is very simple, we just want you to fish out some current or older ## 2- Structure analysis -<<<<<<< HEAD -In collecting a couple of examples for the project structure module we made a mess and lost track of the structure each of the projects. Give it a go and see if you can figure out which patterns do these projects follow. - -[Link to mysterious project 1](https://github.com/qiuyu96/CoDeF/tree/main) - -[Link to mysterious project 2](https://github.com/RoberVega/mysterious_project_2) - -[Link to mysterious project 3](https://github.com/RoberVega/mysterious_project_3) - -[Link to mysterious project 4](https://github.com/cookiecutter/cookiecutter/tree/main) - - -# Assignment 2: code exploration using a debugger - -The last part of this assignment is easy, but very important. One of the most common scenarios is to stumble upon code you have not written yourself. Especially when facing big projects, this might come across as a daunting task. Fortunately, debuggers come to the rescue! One of the possible use cases for debuggers is understanding a new piece of code, especially for studying its intricacies and connections. By introducing strategic breakpoints, one can understand the intended flow of the project and the different parts it is composed of. - -That situation takes us to our next exercise: in the next module, we will focus on testing. But first, we would like you to use a debugger to try to understand how the current testing code actually works, what is being called, etc. -======= Let's practice your ability to navigate projects you have not written yourself. - How would you go about understanding the different components of a project for the first time? - What if you are looking for a specific feature, class, function, _etc._? How would you proceed then? @@ -92,4 +74,9 @@ By understanding we mean a very simple task: prepare a brief general description 1. Can you identify the main folders? Without looking at the code (you can only look inside other folders) what is their purpose? 2. Where is the main script of the package located? ->>>>>>> origin/main + +# Assignment 2: code exploration using a debugger + +The last part of this assignment is easy, but very important. One of the most common scenarios is to stumble upon code you have not written yourself. Especially when facing big projects, this might come across as a daunting task. Fortunately, debuggers come to the rescue! One of the possible use cases for debuggers is understanding a new piece of code, especially for studying its intricacies and connections. By introducing strategic breakpoints, one can understand the intended flow of the project and the different parts it is composed of. + +That situation takes us to our next exercise: in the next module, we will focus on testing. But first, we would like you to use a debugger to try to understand how the current testing code actually works, what is being called, etc. \ No newline at end of file