Rework with pydantic + svg images in documentation by hf-krechan · Pull Request #380 · bo4e/BO4E-python

hf-krechan · 2022-06-07T22:20:55Z

This draft PR should just show an example, how the pydantic approach could look like.

To avoid breaking unit tests I just copied the BO (Business Object) Geschaeftsobjekt and the COM (component) Externe Referenz as well as their unit tests and added the prefix pydantic_ to the file/module names.

Fixes #374

example BO is Geschaeftsobjekt

- ♻refactor Models and data validation - 🚧refactor tests - important note: Now e.g. "2.4" would be accepted as Decimal if the string is parsable (and it will be parsed!) -> a field with e.g. Decimal type hint will always be Decimal. But it can be set/initialized through e.g. a string if the string is parsable into a Decimal. -> change of some test logic is needed

lgtm-com · 2022-06-17T13:02:06Z

This pull request introduces 111 alerts when merging 384358a into 8afe110 - view on LGTM.com

new alerts:

111 for Unused import

- tests, coverage and linting now run without any issues - type_check fails at the moment -> does not find pydantic - next step: Update documentary

Added diagrams for `Geschaeftsobjekt` and `COM`

.github/workflows/tests.yml

docs/uml.py

- python 3.10 ist needed (because of NoneType in test-files) - switch from marschmallow + attr to pydantic

lord-haffi · 2022-06-28T22:45:01Z

Some notes to the PR:

switched from marshmallow + attr / attrs to pydantic
- data validation and de/serialization will be processed by pydantic
  - serialization with <instance>.json(by_alias=True, ensure_ascii=False)
  - deserialization with <class>.parse_raw(<json_string>)
  - note: pydantic comes with some useful types
- now schema-classes aren't neccessary anymore -> removed
- json-schemas for readthedocs are created via .schema_json()
  - note: json-schema files got renamed from <classname>Schema.json to <classname>.json
- important note: Now e.g. "2.4" would be accepted as Decimal if the string is parsable (and it will be parsed!) -> a field with e.g. Decimal type hint will always be Decimal. But it can be set/initialized through e.g. a string if the string is parsable into a Decimal.
auto creation of uml-diagrams and integration with sphinx via graphviz
- note: If you build the docs local via tox -e docs or tox, Graphviz tools (non-python software) need to be installed on your machine and its bin added to PATH (the install software should do this for you). Otherwise it will crash by unknown command dot.
- uml-diagrams are generated for all classes in bo and com via docs/conf.py in sphinx on runtime by calling docs/uml.py -> creates *.dot files in docs/api/dots
- *.dot files can be referenced in docstrings via graphviz directive

hf-kklein

Das sieht ja schonmal ganz nice aus 👍

Ich hätte gerne bei den Grafiken noch zwei,drei Improvements (wenn mit vertretbarem Aufwand machbar):

Können wir das Businessobject/die Komponente, die im Zentrum steht nach ganz oben statt nach ganz unten ziehen oder irgendwie hervorheben?
Können wir (wie in der bisherigen Doku) Business Objects und COMponenten verschieden einfärben?
Können wir die Kardinalitäten an die Pfeil dranschreiben?

hf-kklein · 2022-06-29T05:32:43Z

CONTRIBUTING.md


-All COMponents inherit from `bo4e.com.COM`.
-All Business Objects inherit from `bo4e.bo.Geschaeftsobjekt`.
+All COMponents inherit from `bo4e.com.com.COM`.


Suggested change

All COMponents inherit from `bo4e.com.com.COM`.

All COMponents inherit from `bo4e.com.COM`.

Der Namenspace der Klasse COM ist aber
bo4e.<package>.<module>.<class>
D.h.
bo4e.com.com.COM

vll wäre sowas wie base_com sinnvoll

Meinste? Ich find com com COM ja großartig 😁

CONTRIBUTING.md

docs/uml.py

json_schemas/generate_json_schemas.py

requirements.in

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

lord-haffi · 2022-06-29T09:13:56Z

Können wir das Businessobject/die Komponente, die im Zentrum steht nach ganz oben statt nach ganz unten ziehen oder irgendwie hervorheben?

Das wird schwierig. Ich kann mal versuchen, ob die verschiedenen Layout-Algorithmen irgendwas besser machen, aber manuell etwas zu setzen würde nur gehen, wenn man sich einen eigenen Layout-Algorithmus definiert. Wobei ich auch nichtmal sicher bin, ob man den dann so einfach integrieren kann.

Können wir (wie in der bisherigen Doku) Business Objects und COMponenten verschieden einfärben?

Kann gut sein, dass das geht, ich schau mal.

Können wir die Kardinalitäten an die Pfeil dranschreiben?

Das geht auf jeden Fall - sogar mit verhältnismäßig wenig Aufwand.

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

Added back python versions 3.8 and 3.9 to github CI workflow because `typingx.NoneType` is unnecessary.

lord-haffi · 2022-07-08T13:42:42Z

Since some things changed, an updated list of important things:

switched from marshmallow + attr / attrs to pydantic
- data validation and de/serialization will be processed by pydantic
  - serialization with <instance>.json(by_alias=True, ensure_ascii=False)
  - deserialization with <class>.parse_raw(<json_string>)
  - note: pydantic comes with some useful types
- now schema-classes aren't neccessary anymore -> removed
- json-schemas for readthedocs are created via .schema_json()
  - important note: json-schema files got renamed from <classname>Schema.json to <classname>.json
- important note: Now e.g. "2.4" would be accepted as Decimal if the string is parsable (and it will be parsed!) -> a field with e.g. Decimal type hint will always be Decimal. But it can be set/initialized through e.g. a string if the string is parsable into a Decimal.
auto creation of uml-diagrams and integration with sphinx via plantuml
- note: If you build the docs local via tox -e docs or tox, Plantuml (Java software) will be downloaded automatically via curl into your docs folder. For instance, this means you must have a Java Runtime Environment installed and Java on path in order to build the docs.
- uml-diagrams are generated for all classes in bo and com via docs/conf.py in sphinx on runtime by calling docs/uml.py -> creates *.puml files in docs/api/uml
- these *.puml files are compiled by docs/plantuml.jar into docs/_static/images.
- *.svg files can be included in docstrings via an object tag, e.g. <object data="../_static/images/bo4e/bo/Marktlokation.svg" type="image/svg+xml"></object>

hf-kklein

Nur noch ein paar kleinigkeiten :) Sonst siehts gut aus. Bitte vor dem squashen dem PR/bzw. dem entstehenden Commit auf main noch einen aussagekräftigen Namen geben (also sinngemäß nicht mehr mwe sondern eher: gesamt-umbau+doku)

docs/uml.py

setup.cfg

docs/uml.py

The cache functionality of the network is now clearer and better. Also some minor enhancements, see PR.

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

hf-krechan added 2 commits June 8, 2022 00:15

🎨 Make a MWE with pydantic

efad136

example BO is Geschaeftsobjekt

✅ Add test for pydantic-MWE

3dd5b0a

hf-krechan requested review from hf-kklein and lord-haffi June 7, 2022 22:21

hf-krechan and others added 9 commits June 8, 2022 00:25

➕ Add pydantic dependency

c651331

🎨 Add alias_generator to COM and BO base class

b0f9c71

✅Add test to check camelCase for JSON keys

3949514

🔥 Remove deprecated lines

7e61b1b

🚧wip

aaf5284

🚧wip

d828c98

🔥attrs stuff removed

e4fd81e

🔥pydantic > attrs

fe50e40

lord-haffi and others added 12 commits June 18, 2022 14:33

♻✔Refactor almost done

5fe896a

- tests, coverage and linting now run without any issues - type_check fails at the moment -> does not find pydantic - next step: Update documentary

✨📝Create UML-diagrams on tox -e docs and integrate in sphinx

4ab4dea

🩹Easy fix path separator

9a2b50e

📝Removed Optional from UML-diagrams and added diagrams

597119d

Added diagrams for `Geschaeftsobjekt` and `COM`

♻Updated json-schema auto creation and rebuilt these.

c1845b8

♻📝Added back Hints for JSON-Schemas and Optional type hints

3bec8f3

🚨Fixed mypy errors

e6e6d18

🚨Again some linter / type check fixes destroyed by black

49d3110

💚🚨Fix GitHub Actions and mypy warnings (and removed some console prints)

e77f79c

Merge branch 'main' into mwe-with-pydantic

2f4faa3

💚

5dce733

🩹isort

a899313

hf-kklein reviewed Jun 28, 2022

View reviewed changes

.github/workflows/tests.yml Show resolved Hide resolved

docs/uml.py Show resolved Hide resolved

docs/uml.py Outdated Show resolved Hide resolved

docs/uml.py Outdated Show resolved Hide resolved

lord-haffi added 2 commits June 28, 2022 17:20

🚨💚Added uml.py to the linting CI and fixed linter warnings.

08d957e

Adjusted README-like files

28cf75b

- python 3.10 ist needed (because of NoneType in test-files) - switch from marschmallow + attr to pydantic

Renamed _recursive_helper

8709e47

hf-kklein reviewed Jun 29, 2022

View reviewed changes

Update docs/uml.py

7a3fea0

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

lord-haffi and others added 8 commits June 29, 2022 12:43

Update json_schemas/generate_json_schemas.py

061ad00

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

Update requirements.in

c942ecc

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

Update requirements.in

f595a42

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

Compile uml-diagrams with Plantuml

732de99

💚🚨Make everything linter conform. Added back python v3.8 and v3.9

1f473fd

Added back python versions 3.8 and 3.9 to github CI workflow because `typingx.NoneType` is unnecessary.

💚Adjusted python version issues

fb976f7

🩹isort and type-check (damn you black!)

0641a49

🩹You guessed it. Type-check fix, yes

108b55e

lord-haffi added 3 commits July 8, 2022 15:48

Fixed a comment

2bce5db

Better type ignores on _UMLNetworkABC.model_field_str()

03a32e1

Improved some comments and console prints

10c78fe

lord-haffi marked this pull request as ready for review July 8, 2022 15:32

hf-kklein approved these changes Jul 10, 2022

View reviewed changes

hf-krechan changed the title ~~MWE with pydantic~~ Rework with pydantic + svg images in documentation Jul 11, 2022

lord-haffi and others added 9 commits July 11, 2022 12:05

Compile plantuml files with web service kroki instead of plantuml.jar

d39699a

Some fixes and enhancements

d1b6011

The cache functionality of the network is now clearer and better. Also some minor enhancements, see PR.

Update CONTRIBUTING.md

7a39d95

fix isort

9bcb34f

Update docs/uml.py

34f6326

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

Update tox.ini

a5404a3

Co-authored-by: konstantin <konstantin.klein@hochfrequenz.de>

Merge branch 'main' into mwe-with-pydantic

88c2bc0

Some improvements (see PR)

94c03a5

linter fixes

9af3990

lord-haffi merged commit 63a5ec0 into main Jul 13, 2022

lord-haffi deleted the mwe-with-pydantic branch July 13, 2022 15:35

	All COMponents inherit from `bo4e.com.com.COM`.
	All COMponents inherit from `bo4e.com.COM`.

Conversation

hf-krechan commented Jun 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lgtm-com bot commented Jun 17, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lord-haffi commented Jun 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Some notes to the PR:

Uh oh!

hf-kklein left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hf-kklein Jun 29, 2022

Choose a reason for hiding this comment

Uh oh!

lord-haffi Jun 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hf-krechan Jul 7, 2022

Choose a reason for hiding this comment

Uh oh!

lord-haffi Jul 8, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lord-haffi commented Jun 29, 2022

Uh oh!

lord-haffi commented Jul 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Since some things changed, an updated list of important things:

Uh oh!

hf-kklein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hf-krechan commented Jun 7, 2022 •

edited

Loading

lord-haffi commented Jun 28, 2022 •

edited

Loading

hf-kklein left a comment •

edited

Loading

lord-haffi Jun 29, 2022 •

edited

Loading

lord-haffi commented Jul 8, 2022 •

edited

Loading