Import data from GSD #706

[ziadhany]

This PR adds a new importer for the GSD #706
Signed-off-by: Ziad [email protected]

[TG1999]

Try to extract a reference ID from the reference url, I think a valid id for this example can be SB10198

[TG1999]

@ziadhany please run this importer and add logs for that here, and also run the improve command over this imported data and provide the logs for that

[ziadhany]

Importing data using vulnerabilities.importers.gsd.GSDImporter
JSONDecodeError('Expecting value: line 1 column 2 (char 1)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Extra data: line 1 column 2 (char 1)')
JSONDecodeError('Extra data: line 1 column 2 (char 1)')
JSONDecodeError('Extra data: line 1 column 2 (char 1)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Extra data: line 1 column 2 (char 1)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
JSONDecodeError('Expecting value: line 1 column 1 (char 0)')
Successfully imported data using vulnerabilities.importers.gsd.GSDImporter

Process finished with exit code 0

[ziadhany]

` in SQL comments. Only the formatting feature that removes comments from SQL statements is affected by this regular expression., provided: sqlparse is a non-validating SQL parser module for Python. In sqlparse versions 0.4.0 and 0.4.1 there is a regular Expression Denial of Service in sqlparse vulnerability. The regular expression may cause exponential backtracking on strings containing many repetitions of '\r\n' in SQL comments. Only the formatting feature that removes comments from SQL statements is affected by this regular expression. As a workaround don't use the sqlformat.format function with keyword strip_comments=True or the --strip-comments command line flag when using the sqlformat command line tool. The issues has been fixed in sqlparse 0.4.2.
Inconsistent summary for <Vulnerability: VULCOID-20V>. Existing: Path Traversal in SharpZipLib, provided: SharpZipLib (or #ziplib) is a Zip, GZip, Tar and BZip2 library. Prior to version 1.3.3, a TAR file entry `../evil.txt` may be extracted in the parent directory of `destFolder`. This leads to arbitrary file write that may lead to code execution. The vulnerability was patched in version 1.3.3.
Inconsistent summary for <Vulnerability: VULCOID-20U>. Existing: Path Traversal in SharpZipLib, provided: SharpZipLib (or #ziplib) is a Zip, GZip, Tar and BZip2 library. Starting version 1.3.0 and prior to version 1.3.3, a check was added if the destination file is under destination directory. However, it is not enforced that `destDir` ends with slash. If the `destDir` is not slash terminated like `/home/user/dir` it is possible to create a file with a name thats begins with the destination directory, i.e. `/home/user/dir.sh`. Because of the file name and destination directory constraints, the arbitrary file creation impact is limited and depends on the use case. Version 1.3.3 contains a patch for this vulnerability.
Traceback (most recent call last):
  File "/home/ziad/PycharmProjects/vulnerablecode/vulnerabilities/management/commands/improve.py", line 59, in improve_data
    ImproveRunner(improver).run()
  File "/home/ziad/PycharmProjects/vulnerablecode/vulnerabilities/improve_runner.py", line 40, in run
    inferences = improver.get_inferences(advisory_data=advisory.to_advisory_data())
  File "/home/ziad/PycharmProjects/vulnerablecode/vulnerabilities/models.py", line 403, in to_advisory_data
    affected_packages=[AffectedPackage.from_dict(pkg) for pkg in self.affected_packages],
  File "/home/ziad/PycharmProjects/vulnerablecode/vulnerabilities/models.py", line 403, in <listcomp>
    affected_packages=[AffectedPackage.from_dict(pkg) for pkg in self.affected_packages],
  File "/home/ziad/PycharmProjects/vulnerablecode/vulnerabilities/importer.py", line 208, in from_dict
    affected_version_range = VersionRange.from_string(
  File "/home/ziad/PycharmProjects/univers/venv/lib/python3.10/site-packages/univers/version_range.py", line 118, in from_string
    raise ValueError(f"{vers!r} specifies no version range constraints.")
ValueError: 'vers:gem/' specifies no version range constraints.
Failed to run improver vulnerabilities.improvers.default.DefaultImprover. Continuing...
CommandError: 1 failed!: vulnerabilities.improvers.default.DefaultImprover

[TG1999]

@ziadhany according to your logs the improver has some errors, were you able to investigate or identify the underlying issues.

[ziadhany]

@ziadhany according to your logs the improver has some errors, were you able to investigate or identify the underlying issues.

I am trying to reduce the size of GSD by downloading the zip file so I can easily debug the importer and improve.

[pombredanne]

Based on discussions it seems that GSD has unique vulnerabilities for the Linux kernel that are not available elsewhere and we need to reconsider what to do with these.

[ziadhany]

Based on discussions it seems that GSD has unique vulnerabilities for the Linux kernel that are not available elsewhere and we need to reconsider what to do with these.

ok, I'll work on that this week

[TG1999]

@pombredanne @ziadhany what have we decided on this PR, is it ready for review ?

[pombredanne]

@ziadhany Let's not activate the importer for now for all the reasons we have discussed here and in weekly calls.
Once deactivated (with the suggested modifications) we can then merge.

[pombredanne]

We could merge the PR even if we do not actually run the importer just yet. I would like to merge the code as we may use it in the future.

Suggested change

gsd.GSDImporter,

[ziadhany]

Done

[TG1999]

LGTM! @pombredanne please see if this can be merged ?

[TG1999]

LGTM!