Skip to content

Switch build_dbs.py to boto3, add Pfam-A.clans.tsv auto-download, replace MySQLdb with mysql-connector-python#66

Merged
murphycj merged 9 commits into
masterfrom
copilot/update-build-dbs-and-database
Apr 9, 2026
Merged

Switch build_dbs.py to boto3, add Pfam-A.clans.tsv auto-download, replace MySQLdb with mysql-connector-python#66
murphycj merged 9 commits into
masterfrom
copilot/update-build-dbs-and-database

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 8, 2026

  • Add boto3 and mysql-connector-python to requirements-dev.txt
  • Update bin/build_dbs.py to use boto3 for S3 operations directly (no wrapper functions), credentials from environment
  • Update bin/build_dbs.py to download Pfam-A.clans.tsv from S3 if it doesn't exist locally
  • Update bin/build_dbs.py to use logging instead of print statements
  • Update agfusion/database.py to use mysql-connector-python instead of MySQLdb, with error handling
Original prompt

Update the #file:build_dbs.py file with the following requirements:

  • Have it use the boto3 aws library when download/uploading any files from s3. it can assume credentials come from environemnt variables. boto3 should only be a dev dependency in #file:requirements-dev.txt
  • Have it download download the 'Pfam-A.clans.tsv' file from the s3 bucket if it does not exist locally
  • Update #file:database.py to use mysql-connector-python instead. It should have that dependency installed in #file:requirements-dev.txt
    Create a pull request.

The user has attached the following file paths as relevant context:

  • requirements-dev.txt
  • agfusion/database.py
  • bin/build_dbs.py
  • .github/instructions/python.instructions.md
  • .github/copilot-instructions.md

Created from VS Code.

…S3, switch to mysql-connector-python

Agent-Logs-Url: https://github.com/murphycj/AGFusion/sessions/3caeb2ae-0ca5-4d89-ba75-409b13fdd2b1

Co-authored-by: murphycj <3837196+murphycj@users.noreply.github.com>
Copilot AI changed the title [WIP] Update build_dbs.py to use boto3 and database.py for mysql-connector-python Switch build_dbs.py to boto3, add Pfam-A.clans.tsv auto-download, replace MySQLdb with mysql-connector-python Apr 8, 2026
Copilot AI requested a review from murphycj April 8, 2026 20:37
@murphycj
Copy link
Copy Markdown
Owner

murphycj commented Apr 8, 2026

@copilot get rid of the wrapper functions around the boto download and upload functions. Just use boto directly. Also, use the logging library instead of print statements for logging

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 8, 2026

@copilot get rid of the wrapper functions around the boto download and upload functions. Just use boto directly. Also, use the logging library instead of print statements for logging

Done in ebbee3e. Removed the s3_key_exists, s3_upload, and s3_download wrapper functions — boto3 is now called directly inline. Replaced all print statements with logger.info calls using the standard logging module.

@murphycj murphycj marked this pull request as ready for review April 9, 2026 01:17
@murphycj murphycj merged commit 94eed6f into master Apr 9, 2026
3 checks passed
@murphycj murphycj deleted the copilot/update-build-dbs-and-database branch April 9, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants