Testing Procedures

Set up branch for testing

Clone the repository and check out the branch to be tested. Assume the branch to be tested is called feature/new and the final desired result is for feature/new to be merged into the development branch. We need to merge locally to test.

Important: Always create a fresh clone in a new directory

git clone git@github.com:esmf-org/esmf-profiler.git
cd esmf-profiler
git checkout feature/new
git checkout development
git merge feature/new

Local Install

1. Ensure that the install_dependencies.sh and install.sh have executable permissions

chmod +x ./install_dependencies.sh 
chmod +x ./install.sh

2. Execute install_dependencies.sh

./install_dependencies.sh

• ./dependencies/INSTALL/babeltrace2-2.0.4 exists
• ./dependencies/INSTALL/babeltrace2-2.0.4/bin/babeltrace2 executes and produces a usage message

3. Execute install.sh

./install.sh

• directory ./venv is created
• ./venv/bin/activate has PYTHONPATH and LD_LIBRARY_PATH correctly set at bottom of script and those paths exist

3. Activate the virtual environment.

source ./venv/bin/activate

4. Install the esmf-profiler using pip editable install.

pip install -e .[test] 

• version tag of esmf-profiler python package is as expected

5. Run pytests

python -m pytest 

• all pytest tests pass

6. Run esmf-profiler without arguments

esmf-profiler

• esmf-profiler usage message is displayed when run with no command line args

7. Generate a test profile and view the result using the local server flag

esmf-profiler -t ./tests/fixtures/test-traces/atm-ocn-concurrent -n "testname" -o "testoutput" -s

• Verify the generated profile is viewable on a browser using port 8000
• Verify exiting the app kills the local server process cleanly

Docker Image Build and Run

1. Build the image Important: use --no-cache argument to rebuild all layers

docker build -t esmf-profiler-image . --no-cache

• docker build completes without error

2. Process a test trace using the newly built image

docker run -it -v $(pwd)/testout:/home/testout esmf-profiler-image esmf-profiler -t /home/esmf-profiler/tests/fixtures/test-traces/atm-ocn-concurrent -n 'automatedtest' -o /home/testout

• testout directory contains static site files
• testout/data has load_balance.json and site.json files

Push Profile to GitHub

1. Follow procedures for local install

2. Create a new empty repository on GitHub or use an existing one. Enable Github Pages for the repository.

3. Process a trace and push the results to the remote repository.

esmf-profiler -t tests/fixtures/test-traces -n 'profilepushtest' -o ./out -p git@github.com:<repo-url>

• static site is committed and pushed to remote repo (navigate to GitHub to confirm commit is there)
• static site is in the correct place in the directory structure (username/name)
• static site GUI is displayed as expected at the URL

4. Repeat push

esmf-profiler -t tests/fixtures/test-traces -n 'profilepushtest' -o ./out -p git@github.com:<repo-url>

• request to repeat push should succeed

Profile Site UI Testing

Note: These tests need to be repeated. (1) site accessed from local file system and (2) site pushed to GitHub.io.

1. Generate profile site.

esmf-profiler -t ./tests/fixtures/test-traces/atm-ocn-concurrent -n myname -o out

Note: for local testing, do not pass the --push option. For remote testing add the --push option above.

2. Perform these UI checks in Chrome, Firefox, and IE:

   • the root "index.html" displays the site in the browser
   • the site title is correctly displayed at the top of the page and in the browser tab or title bar
   • the timestamp is correctly displayed at the top of the page
   • the version number displayed in upper left matches installed version of esmf-profiler
   • the menu items (currently only Load Balance) work as expected
   • left menu expand/collapse option works correctly
   • drill down functionality of Load Balance chart works:
     • drill down until reaching max depth and verify error message is displayed.
     • breadcrumb functions correctly when navigating up and down the levels and jumping between depth levels.
   • Select All / Select None function properly at several different levels
   • turning on/off individual items in the legend works and chart updates as expected
   • chart image download options work as expected (PNG, JPEG, etc.)
   • full screen chart option works as expected

3. Open Javascript console in Chrome and reload page several times.

   • no Javascript errors are produced and any warnings are understood