[WIP] Fix schema by hsd-dev · Pull Request #359 · robust-rosin/robust

hsd-dev · 2020-09-09T11:10:07Z

Fixes most of the issues with the schema.

Though this doesn't allow automating fixing the bug files in any way. Porting to yamale. See #355

gavanderhoorn · 2020-09-09T15:54:10Z

Ha, CI is actually working:

/scripts/yamale.bug:33:13: [error] no new line character at the end of file (new-line-at-end-of-file)

;)

hsd-dev · 2020-09-09T16:10:38Z

YAML schema seems to be complete. Will run the validator on the bugs and will get the most commonly occurring values.

hsd-dev · 2020-09-09T16:16:51Z

scripts/robust.yaml

+system: str(min=1, max=40)
+severity: enum("error", "warning", "convention-violation", "bad-smell",
+            "minor-issue", "not-a-bug")
+links: list(str())


TODO: Fix this

added a preliminary check for URLs in 9e9be82, but better solutions welcome

hsd-dev · 2020-09-09T16:17:45Z

scripts/robust.yaml

+    task: enum("perception", "localization", "planning", "manipulation",
+            "human-robot interaction", "simulation", "diagnostics", "SLAM",
+            "N/A")
+    subsystem: str(min=1)


Shouldn't this be an enum?

you mean the subsystem. I think we should definitely make sure that consistent names and spelling is used, otherwise, I think it is of secondary importance, whether it is an enum or not. If you make it enum it is easier for you to detect divergences. The cost is that the scheme has to be changed whenever a new value pops up. I think this is a price that is entirely acceptable. So if you prefer to let yamale control it, just go ahead.

scripts/robust.yaml

hsd-dev · 2020-09-09T16:20:09Z

scripts/yamale_validator.py

+try:
+    yamale.validate(schema, data)
+    print('Validation success! 👍')
+except yamale.YamaleError as e:


Handle errors in each field

gavanderhoorn · 2020-09-30T18:13:56Z

Ping @ipa-hsd: could you update us on the status of this PR?

hsd-dev · 2020-10-07T14:08:07Z

Command:

find ../ -name "*.bug" | xargs -I {} ./validate.sh {}

Example out of the script (it checks for all bug files at once):

Error validating data '../universal_robot/fc95a19/fc95a19.bug' with './robust.yaml'
	
	bug.package: 'ros-industrial/universal_robot/ur_driver' is not a list.
	bug.languages: 'None' is not a list.
	bug.time-reported: '2014-10-10 (02:34)' is not a date.
	bug.time-reported: '2014-10-10 (02:34)' is not a null.
	bug.trace: 'None' is not a str.
	fix.repo: 'https://github.com/ros-industrial/universal_robot/' is not a regex match.
	fix.languages.0: 'Python' not in ('python', 'cmake', 'C++', 'package.xml', 'launch.xml', 'msg', 'srv', 'xacro', 'urdf', 'robot specific')
	fix.time: '2014-11-03 (16:30)' is not a date.
	fix.time: '2014-11-03 (16:30)' is not a null.
	time-machine: Required field missing
	bugzoo: Required field missing

hsd-dev · 2020-10-13T10:37:25Z

@wasowski the new commits added addresses #370

ChrisTimperley · 2020-10-14T15:29:21Z

scripts/yamale_validator.py

+        print("Error validating data '%s' with '%s'\n\t" % (result.data, result.schema))
+        for error in result.errors:
+            print('\t%s' % error)
+    exit(1)


I think that this should be sys.exit

hsd-dev · 2020-10-15T08:23:51Z

The taxonomy notes the time is in the format 2017-12-31 (23:59). Changed the time to Z-notation UTC timestamp in 56c54eb. With the earlier format the validation fails.

For my ref:

awk '/time:/ && /\)$/ {gsub(/\(|\)/,"");str=$2" "$3; gsub(str, $2"T"$3":00Z")}1' "$file"

gavanderhoorn · 2020-10-15T08:25:40Z

The timestamps should actually be in ISO 8601 format.

I believe what you have (now) is in that format, but just wanted to mention the "official" name.

gavanderhoorn · 2020-10-15T09:32:19Z

Note btw: I'm not sure you can just use the awk command you have to rewrite all timestamp to UTC time like that. If the author of the bug description file is in another timezone, you'll have to convert the timestamps to UTC (ie: subtract delta-t from UTC, etc).

Just changing the format is not going to result in correct timestamps.

Having incorrect timestamps will lead to the time-machine using incorrect rosdistro state, etc.

scripts/yamale_validator.py

ChrisTimperley · 2020-10-15T14:08:44Z

It's probably worth adding a particular version of yamale to scripts/requirements.txt. E.g., yamale==3.0.4.

ChrisTimperley · 2020-10-15T14:11:32Z

scripts/yamale_validator.py

@@ -0,0 +1,18 @@
+# Import Yamale and make a schema object:
+import yamale
+schema = yamale.make_schema('./robust.yaml')


This will prevent the script from being run outside the scripts directory. To allow the script to be used anywhere, you could add:

import os dir_here = os.path.dirname(__file__) schema_path = os.path.join(dir_here, 'robust.yaml') data_path = os.path.join(dir_here, 'yamale.bug')

scripts/robust.yaml

ChrisTimperley · 2020-10-15T14:28:29Z

scripts/validate.sh

-SCHEMA="${DIR}/robust.json"
 echo Checking schema compliance for $1
-any-json --input-format=yaml $1 | ajv --verbose --errors=text -s ${SCHEMA} -d /dev/stdin 2>&1 >/dev/null | sed "s/\/dev\/stdin/File $1/"
+python3 yamale_validator.py $1


Update to allow script to be executed from outside of the scripts directory.

Should we go "all out" and add argparse support?

Then add an arg to specify the location of the schema.

ChrisTimperley · 2020-10-15T14:51:15Z

scripts/robust.yaml

+---
+
+_bug:
+    phase: enum("build", "startup", "runtime", "does not apply")


Update to latest values: #361

trying out yamale though

- added date validator - included time-machine and bugzoo fields in sample bug

not the best solution, since difficult to generalize URLs

- added date validator - included time-machine and bugzoo fields in sample bug

not the best solution, since difficult to generalize URLs

hsd-dev · 2020-10-19T08:10:15Z

As per #376 (comment), dropped 0c46707

hsd-dev · 2020-10-20T08:12:15Z

@gavanderhoorn @ChrisTimperley please take a look now.

kobuki/e964bbb/e964bbb.bug

gavanderhoorn · 2020-10-21T16:05:18Z

kobuki/e964bbb/e964bbb.bug

-  pull-request: null
-  license: ['BSD']
-  fix-in: ['kobuki_driver/src/driver/diff_drive.cpp']
+  pull-request: N/A


Just a question: we don't use null any more for this then? Or was that too ambiguous (as in: could also mean we just haven't bothered to look for one)?

I am not sure in general, but for this specific case, the commit was not part of any PR. So N/A might be more suitable here.

gavanderhoorn · 2020-10-21T16:10:36Z

scripts/validate.sh

+VALIDATOR="${DIR}/yamale_validator.py"
 echo Checking schema compliance for $1
-any-json --input-format=yaml $1 | ajv --verbose --errors=text -s ${SCHEMA} -d /dev/stdin 2>&1 >/dev/null | sed "s/\/dev\/stdin/File $1/"
+python3 ${VALIDATOR} $1


Does our CI setup guarantee Python 3 is installed?

I would suggest to add that, just so we keep things working/consistent.

Just checked and the current .travis.yml actually specifies 2.7.

gavanderhoorn · 2020-10-21T16:11:07Z

scripts/yamale.bug

+  bug-commit: 4f317228c5d5c5bfc96f3f0dfa692bc4b93dcc43
+  fix-commit: b96bf672a718b9f0c9694e4314283e385ba96231
+  fork-urls:
+    - https://github.com/robust-rosin/mavros


This is a "test bug", correct? Should we rename it to something like yamale_test.bug?

We could even delete this. I haven't been careful with this one. The kobuki bug in this PR is more complete.

gavanderhoorn

Other than the inline comments, this looks OK.

it is not complete, was just an initial test to try yamale

hsd-dev · 2020-10-22T09:15:00Z

addressed the inline comments by @gavanderhoorn. Merging this PR.

ChrisTimperley · 2020-10-22T14:12:20Z

Bit of a pet peeve, sorry, but remember to squash and rebase next time so that we maintain a sensible commit history on master 👍🏻

hsd-dev force-pushed the fix-schema branch from 87a8fa5 to 15e1f72 Compare September 9, 2020 15:47

hsd-dev force-pushed the fix-schema branch from 15e1f72 to 892fb4d Compare September 9, 2020 16:08

hsd-dev force-pushed the fix-schema branch from 892fb4d to 009c275 Compare September 9, 2020 16:12

hsd-dev commented Sep 9, 2020

View reviewed changes

scripts/robust.yaml Outdated Show resolved Hide resolved

hsd-dev commented Sep 9, 2020

View reviewed changes

scripts/robust.yaml Show resolved Hide resolved

hsd-dev commented Sep 9, 2020

View reviewed changes

ChrisTimperley reviewed Oct 14, 2020

View reviewed changes

ChrisTimperley reviewed Oct 15, 2020

View reviewed changes

scripts/yamale_validator.py Show resolved Hide resolved

ChrisTimperley reviewed Oct 15, 2020

View reviewed changes

scripts/robust.yaml Show resolved Hide resolved

ChrisTimperley reviewed Oct 15, 2020

View reviewed changes

hsd-dev added 7 commits October 18, 2020 22:42

converted to arrays

02d3404

[WIP] fixes issues with the json schema

7cf0405

trying out yamale though

porting schema to yamale

5a2b019

update validator and bug

c0f3886

- added date validator - included time-machine and bugzoo fields in sample bug

validate links

8d28830

not the best solution, since difficult to generalize URLs

include for hash

d1c727d

use yamale in validate.sh

89d455f

hsd-dev added 5 commits October 18, 2020 22:42

test for parsing yaml

050bb96

clean bug file

0980d08

added fault and failure codes in schema

bf1ddad

update validator and bug

dfefc15

- added date validator - included time-machine and bugzoo fields in sample bug

validate links

0c66f0a

not the best solution, since difficult to generalize URLs

hsd-dev force-pushed the fix-schema branch from e715bf3 to 8646225 Compare October 18, 2020 21:24

hsd-dev mentioned this pull request Oct 19, 2020

[WIP] Fix cob #376

Closed

hsd-dev added 3 commits October 19, 2020 10:04

included locomotion in task schema

16a8b58

allow running script from outside folder

f67bb23

updated schema as per issues/361

8d1c78e

hsd-dev force-pushed the fix-schema branch from 59e461e to 8d1c78e Compare October 19, 2020 08:05

hsd-dev added 3 commits October 19, 2020 10:11

added encoding

ff8ced9

run bash script from anywhere

673d8c5

added yamale to requirements.txt

e55f482

gavanderhoorn reviewed Oct 20, 2020

View reviewed changes

kobuki/e964bbb/e964bbb.bug Show resolved Hide resolved

ChrisTimperley approved these changes Oct 21, 2020

View reviewed changes

gavanderhoorn reviewed Oct 21, 2020

View reviewed changes

gavanderhoorn approved these changes Oct 21, 2020

View reviewed changes

hsd-dev added 2 commits October 22, 2020 11:08

added python 3.6 to travis

7971f5f

deleted yamale.bug

d5dddb8

it is not complete, was just an initial test to try yamale

hsd-dev merged commit deea65e into robust-rosin:master Oct 22, 2020

hsd-dev deleted the fix-schema branch October 22, 2020 09:18

Conversation

hsd-dev commented Sep 9, 2020

Uh oh!

gavanderhoorn commented Sep 9, 2020

Uh oh!

hsd-dev commented Sep 9, 2020

Uh oh!

hsd-dev Sep 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gavanderhoorn commented Sep 30, 2020

Uh oh!

hsd-dev commented Oct 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hsd-dev commented Oct 13, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hsd-dev commented Oct 15, 2020

Uh oh!

gavanderhoorn commented Oct 15, 2020

Uh oh!

gavanderhoorn commented Oct 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ChrisTimperley commented Oct 15, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hsd-dev commented Oct 19, 2020

Uh oh!

hsd-dev commented Oct 20, 2020

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gavanderhoorn left a comment

Choose a reason for hiding this comment

Uh oh!

hsd-dev commented Oct 22, 2020

Uh oh!

ChrisTimperley commented Oct 22, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

hsd-dev Sep 9, 2020 •

edited

Loading

hsd-dev commented Oct 7, 2020 •

edited

Loading

gavanderhoorn commented Oct 15, 2020 •

edited

Loading