Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions .github/workflows/generate-model.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,21 @@ jobs:
if: steps.filter.outputs.parsing_required == 'true'
uses: ts-graphviz/setup-graphviz@v1

- name: Setup Python
uses: actions/setup-python@v5
- name: Install uv
if: steps.filter.outputs.parsing_required == 'true' || steps.filter.outputs.test_case_parsing_required == 'true'
uses: astral-sh/setup-uv@v7
with:
python-version: '3.11'
enable-cache: true

- name: Install Python
if: steps.filter.outputs.parsing_required == 'true' || steps.filter.outputs.test_case_parsing_required == 'true'
run: uv python install 3.11

- name: Install python requirements
working-directory: ./csv_parser
if: steps.filter.outputs.parsing_required == 'true' || steps.filter.outputs.test_case_parsing_required == 'true'
run: pip install -r ./requirements.txt
run: uv sync

- name: Clean up old generated schemas
working-directory: ./src/main/resources
Expand All @@ -66,19 +72,19 @@ jobs:
- name: Run csv_parser to generate schemas.yaml
working-directory: ./csv_parser
if: steps.filter.outputs.parsing_required == 'true'
run: python workflow.py --stage output_schemas_yaml
run: uv run python workflow.py --stage output_schemas_yaml

- name: Run csv_parser and collect OpenAPI & JSON Schemas
working-directory: ./csv_parser
if: steps.filter.outputs.parsing_required == 'true'
run: python workflow.py --stage parser_and_mv
run: uv run python workflow.py --stage parser_and_mv

- name: Run test_case_generator
working-directory: ./csv_parser
if: steps.filter.outputs.test_case_parsing_required == 'true'
run: |
rm -r ./out/test-cases || true
python workflow.py --stage test_case_parser
uv run python workflow.py --stage test_case_parser

- name: Collect schemas.yaml and copy it to json_schema2xsd
working-directory: ./csv_parser
Expand Down
1,074 changes: 737 additions & 337 deletions converter/uv.lock

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion csv_parser/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,5 @@ out/**/*.schema.json
out/**/*.openapi.yaml

# virtual env
env/
env/
.venv/
1 change: 1 addition & 0 deletions csv_parser/.python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11
14 changes: 7 additions & 7 deletions csv_parser/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# Installation
```bash
pip install -r requirements.txt
uv sync
```
For the UML generator (run automatically within the CSV parser), in addition to the `requirements.txt`, you need to install graphviz on your execution environment https://graphviz.org/
For the UML generator (run automatically within the CSV parser), in addition to the Python dependencies, you need to install graphviz on your execution environment https://graphviz.org/

# Usage
## CSV Parser
```bash
# Params to specify the sheet and version
# by default, integrate uml generation process
python csv_parser.py -s RC-DE -v 0.5
python csv_parser.py --sheet RC-DE --version 0.5
uv run python csv_parser.py -s RC-DE -v 0.5
uv run python csv_parser.py --sheet RC-DE --version 0.5
# Defaults to RC-EDA and today (YY.MM.DD)
python csv_parser.py
uv run python csv_parser.py
```

## Full pipeline
Expand All @@ -28,8 +28,8 @@ act -s GITHUB_TOKEN="$(gh auth token)"
UML generator is called by default in the `csv_parser.py` but you may want to run it directly.
```bash
# Params to specify model and version
python uml_generator.py -m RC-EDA -o cisu -v 1.12
python uml_generator.py --model RC-EDA --obj cisu --version 1.12
uv run python uml_generator.py -m RC-EDA -o cisu -v 1.12
uv run python uml_generator.py --model RC-EDA --obj cisu --version 1.12
```

# Automatic nomenclature & model push
Expand Down
6 changes: 3 additions & 3 deletions csv_parser/auto.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ TECHNICAL_MODELS_FILE="models/model-technical.xlsx"
TRACKING_BRANCH_NAME="auto/model_tracker"
DATE=$(date +'%y.%m.%d %H:%M')
LOG_FILE="cron.log"
PYTHON_CMD="/Users/romainfouilland/code/envs/all/bin/python"
PYTHON_CMD="uv run python"

# Enable printing of each command
set -x
Expand All @@ -33,7 +33,7 @@ update_nomenclatures() {
else
echo "Changes detected in $NOMENCLATURE_FOLDER, running nomenclatures generation..."
cd ../nomenclature_parser
${PYTHON_CMD[@]} nomenclature_parser.py || (git stash && exit 1)
${PYTHON_CMD} nomenclature_parser.py || (git stash && exit 1)
git add .
fi
}
Expand All @@ -44,7 +44,7 @@ update_test_cases() {
echo "No changes in $TEST_CASE_SOURCES_FOLDER, skipping test cases generation..."
else
echo "Changes detected in $TEST_CASE_SOURCES_FOLDER, running test cases generation..."
${PYTHON_CMD[@]} workflow.py --stage test_case_parser || (git stash && exit 1)
${PYTHON_CMD} workflow.py --stage test_case_parser || (git stash && exit 1)
git add .
fi
}
Expand Down
26 changes: 26 additions & 0 deletions csv_parser/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
[project]
name = "csv-parser"
version = "0.1.0"
description = "CSV Parser service for SAMU-Hub-Modeles"
readme = "README.md"
requires-python = ">=3.9"
dependencies = [
"pandas>=2.0.0",
"openpyxl>=3.0.0",
"pyyaml>=6.0.2",
"jsonpath-ng>=1.7.0",
"python-docx>=1.0.0",
"argparse>=1.4.0",
"graphviz>=0.20.0",
]

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = "-v"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
7 changes: 0 additions & 7 deletions csv_parser/requirements.txt

This file was deleted.

Loading
Loading