JSON format support

.github/workflows/codeql-analysis.yml

@@ -0,0 +1,71 @@
+# For most projects, this workflow file will not need changing; you simply need
+# to commit it to your repository.
+#
+# You may wish to alter this file to override the set of languages analyzed,
+# or to provide custom queries or build logic.
+name: "CodeQL"
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: [master]
+  schedule:
+    - cron: '0 0 * * 3'
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: ubuntu-latest
+
+    strategy:
+      fail-fast: false
+      matrix:
+        # Override automatic language detection by changing the below list
+        # Supported options are ['csharp', 'cpp', 'go', 'java', 'javascript', 'python']
+        language: ['python']
+        # Learn more...
+        # https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v2
+      with:
+        # We must fetch at least the immediate parents so that if this is
+        # a pull request then we can checkout the head.
+        fetch-depth: 2
+
+    # If this run was triggered by a pull request event, then checkout
+    # the head of the pull request instead of the merge commit.
+    - run: git checkout HEAD^2
+      if: ${{ github.event_name == 'pull_request' }}
+
+    # Initializes the CodeQL tools for scanning.
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v1
+      with:
+        languages: ${{ matrix.language }}
+        # If you wish to specify custom queries, you can do so here or in a config file.
+        # By default, queries listed here will override any specified in a config file. 
+        # Prefix the list here with "+" to use these queries and those in the config file.
+        # queries: ./path/to/local/query, your-org/your-repo/queries@main
+
+    # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
+    # If this step fails, then you should remove it and run the build manually (see below)
+    - name: Autobuild
+      uses: github/codeql-action/autobuild@v1
+
+    # ℹ️ Command-line programs to run using the OS shell.
+    # 📚 https://git.io/JvXDl
+
+    # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
+    #    and modify them (or add more) to build your code if your project
+    #    uses a compiled language
+
+    #- run: |
+    #   make bootstrap
+    #   make release
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v1

README.md

@@ -1,20 +1,51 @@
-NIST CVE library search engine.
-Provide your project's list of software packages, libraries, and any module used to create your product.
-This tool will use your list to search tousands of NIST CVE entries to find any known issues.
-Knowledge is half the battle, so use this to automate the search for software items that could have outstanding 
-issues needed a patch.
-
-How does it wor? It is a simple python script used to parse your provided list of software modules (packages, libraries and so on) 
-against a database of known software vulnerabilities.
-The vulnerabilities are the NIST provided CVE issues. The script download.xml gathers an updated database from NIST
-and stores it locally.
-Then run the cli.py script with your software to parse through the entire database.
-The output is an XML file showing the matching CVE's.
-
-Fair warning it uses some specific python libraries so your are BEST served by creating a local directory with
-a virtual python envionrment to protect your machine. Trust me.
-
-To get this working you will need to download a copy of the NVD here: https://nvd.nist.gov/download.cfm#CVE_FEED
-and put the xml files in the dbs folder.
-
-run package-cve-lookup -h for more information on using the command line interface.
+Welcome to the [CVE Scanner](https://github.com/ptdropper/CVE-Scanner-for-your-SW-BOM) wiki! 
+
+What is CVE-scanner?
+====================
+This project provides a way that you can manage the risk inherited by using open source and third party source projects. This provides you with intelligent Software Composition Analysis to identify and reduce risk.
+
+Inputs from your project
+========================
+The project is a python based NIST-CVE library search engine for use with your own custom Software Bill of Materials (SBOM) input file. This is ideal for projects where you can create a text file of your SBOM as input to the tool. The output will be all CVE identifiers of potential risks. The library from NIST is tens of thousands of entries, and this tool does the work of searching for your specific packages of interest. 
+
+HOW TO:
+=======
+- Create an ascii text input file holding package names and versions of interest.
+- Input data file contains the triplet "vendor-product-version" with dashes.
+- Must match on all 3 to decide to report the CVE.
+- Lines with a leading hash/pound symbol are ignored.
+
+Example for typical open source packages where there is no vendor so set the vendor value to match the product name.
+```sh
+libssh-libssh-1.0
+linux_kernel-linux_kernel-4.9
+microsoft-home_server-2003
+php-php-5.4.3
+```
+Whitelist to ignore specific CVEs
+=================================
+Next is an optional whitelist file you can create. The whitelist is referred to as the "ignore list" in the python sources. 
+The ignore list content is based on your analysis of the reported CVE's affecting your project. As you review CVE descriptions and details you may find that some of the CVEs do not apply in your product. Then copy those entries into the ignore list file and optionally provide some message to yourself to explain why the CVE does not apply. These ignored CVE's will show up with the marking < skipped > in the report so you are aware they have been analyzed.
+
+Example ignore list file content
+================================
+```sh
+##CVE-2015-7697 does not apply because the product does not use feature foo which is the trigger for this issue.
+CVE-2015-7697
+```
+
+Usage: ./json_cve_parser.py ./my_input_input -i ./my_ignore_list
+Example: the shell script "check_spacecom_json.sh" calls 
+
+./json_cve_parser.py ./spacecom_input -i ./spacecom_ignore_list
+
+The parser is json_cve_parser.py, accepts input file, reads each line, searches the database for that triplet, and if that product tripet is related to a CVE number write the CVE number and summary text to the output file. If that CVE is in the ignore list, then indicate that the CVE is marked with the prefix < skipped >.
+
+So there is more to the process as is captured in the script scan_for_vulnerabilities_json.sh. This script is the real entry point for the process which can be run on the command line, in a cron job, or hooked into Jenkins. THe file performs the required database download from NIST, starts a local python environment using the "activate" command, creates a new directory that has a unique incremental build number, and finally calls the shell script "check_spacecom_json.sh" to execute the process.
+
+Jenkins
+=======
+For a daily build process use the script scan_for_vulnerabilities_json.bat as the hook into your Jenkins build machine. It will call the "scan_for_vulnerablities_json.sh" script in a bash session.
+
+Finally the output directories will continue to grow in number and daily changes to the NIST database will be reflected in the output files in those output directories. One tool to use to verify you can observe changes is to use the Git SCM tool to track every one of the output files. Each day copy the latest over the top of the previous iteration and let the Git diff feature show you what changed. Managing vulnerabilities then means managing these changes.
+

check_com_3_json.sh

@@ -0,0 +1 @@
+./json_cve_parser.py space_com_3_input -i space_com_3_ignore_list

check_dosetrack_json.sh

@@ -0,0 +1,2 @@
+./json_cve_parser.py dosetracklink.input -i ignorecvelist
+

check_spacecom2_wb45_json.sh

@@ -0,0 +1,2 @@
+# -*- coding: utf-8 -*-
+./json_cve_parser.py ./spacecom2_wb45_input -i ./spacecom2_wb45_ignore_list

check_spacecom_json.sh

@@ -0,0 +1,2 @@
+# -*- coding: utf-8 -*-
+./json_cve_parser.py ./spacecom_input -i ./spacecom_ignore_list