Drug Design Data Resource CELPP Runner

Drug Design Data Resource is a suite of software to enable filtering of new entries from wwpdb and scoring of CELPP participant submissions. This package runs on D3R servers to host the CELPP challenge. Participants in CELPP who use the D3R-provided workflow template, CELPPade, will need to install this package as it uses some shared scripts.

For more information on this package or how to begin participating in CELPP, please visit our wiki page:

Compatibility

> - Works with Python 2.6, 2.7

Dependencies

> - argparse > - lockfile > - psutil > - biopython > - xlsxwriter > - ftpretty > - NCBI > Blast > (needed by blastnfilter.py) > - rdkit (needed by blastnfilter.py and > proteinligprep.py) > - schrodinger (needed by > proteinligprep.py, glidedocking.py, chimera_proteinligprep.py & > vinadocking.py) > - babel (needed by > chimera_proteinligprep.py) > - Autodock Vina & MGL > Tools (needed by > vinadocking.py) > - OpenEye Python with > License > (needed by blastnfilter.py symmetry filter and by evaluate.py)

Installation

pip install d3r

Usage

Run

celpprunner.py --help

License

See LICENSE.txt

Bugs

Please report them here

Acknowledgements

• This work is funded in part by NIH grant 1U01GM111528 from the National Institute of General Medical Sciences
• This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

1.11.3 (2019-02-26)

• Additional fix for ftp connection refused Issue #181

1.11.2 (2019-01-16)

• Fix for failed test in 1.11.1

1.11.1 (2019-01-16)

• Email participant when submission is empty. Issue #197
• Allow spaces in passwords in ftp config file. Issue #198

1.11.0 (2018-09-07)

• Fixed minor bug where unit tests for molfilevalidator.py failed when valid openeye license was available. Issue #177
• Added retry to external submission download to prevent complete failure of download in event of network hiccup Issue #181
• Fixed bug where median value incorrectly calculated in evaluate.py. Instead of averaging middle two values for case of even number of elements, the old code just chose the latter value. Issue #183
• Fixed bug where median value incorrectly calculated in post_evaluation.py. Instead of averaging middle two values, the old code just chose the latter value. Issue #184
• Updated BlastNFilterSummary class (used by celppreports.py) to calculate number of targets found by counting target .txt files if value is not in summary.txt file Issue #187
• Added call to website REST service in EvaluationTask and PostEvaluationTask to persist evaluation results to website. This change added a new flag --websiteserviceconfig that requires a configuration file. Issue #188
• Fixed bug in evaluate.py wait_and_check() function which caused it to prematurely give up on an alignment. Issue #189
• Changed evaluate.py so it now writes final.log directly to output directory. Issue #190
• Moved code under if __main__ into main() function to facilitate testing. Issue #191
• Add RMSD.csv to files uploaded in EvaluationTask get_uploadable_files(). Issue #192
• Changed logging in evaluate.py to use module logger instead of root logger Issue #194

1.10.0 (2018-01-09)

• Breaking change Added new flag to celpprunner.py --smtpconfig that lets a user specify smtp configuration information. At the same time removed --replyto --smtp, and --smtpport since those options would be confusing to have also in place. Issue #166
• molfilevalidator.py has new command line option --excludedir to let caller ignore directories when looking for mol files. Issue #167
• Fixed bug where a network hiccup raised an exception during DataImportTask causing celpprunner.py to exit prematurely. Issue #168
• Improved error message output when a participant uploads a malformed challenge data package. Issue #169
• Version of D3R is now output in post evaluation summary email. Issue #172
• Fixed bug where celpprunner.py was not sending an email to people in --summaryemail list if a task fails. Issue #171
• Post evaluation summary email now outputs lines with NA values for any submissions that failed Issue #130

1.9.2 (2017-10-30)

• Added molfilevalidator.py to validate D3R submission tarfiles. Issue #165
• evaluation.py now generates RMSD.json which is a JSON version of RMSD files. Issue #143
• Added a fix to deal with out of memory errors encountered in blastnfilter stage. Issue #5

1.9.1 (2017-08-21)

• Fixed bug where Apo targets not getting pocket center correctly defined Issue #151

1.9.0 (2017-06-23)

• EvaluationTask now records evaluate.py task exit code in a file Issue #134
• Symmetry filter added to blastnfilter.py Issue #145
• In RMSD.txt,RMSD.csv renamed Medium to Median and swapped values for Maximum and Minimum Issue #144
• Added note about new values in parenthesis in individual results email Issue #142

1.8.0 (2017-05-18)

• EvaluationTask modified to pass path to blastnfilter task to evaluate.py Issue #139
• Median RMSD added to post_evaluation.py outputs Issue #136
• Added histogram of RMSD scores to post_evaluation.py outputs Issue #137
• Continuously output the analysis result into the pickle csv and txt files Issue #133
• Align the docked complex using the binding site alignment for each of the crystal template and calculate the RMSD, if the binding site alignment failed, then the whole protein alignment will be applied
• Improve the extraction step in the evaluate.py to ensure the RMSD calculating was only applied to the docked ligand but not all others ligand like solvents or co-factors
• Add the ligand center calculation step in the evaluate.py to output the distance of the docked ligand with the crystal ligand. Also calculate the distance between the original LMCSS ligand center with the crystal ligand center
• Update the genchallengedata.py to extract Apo chain which is closed to the LMCSS ligand Issue #135

1.7.3 (2017-04-11)

• requirements.txt and setup.py modified to require biopython v1.6.8, which is the last version compatible with python2.6

1.7.2 (2017-04-11)

• Bug fix, evaluation of a submission will not fail if one candidate in that submission fails.

1.7.1 (2017-03-27)

• Bug fix, evaluation task will not fail if call to external script fails. Issue #129

1.7.0 (2017-03-21)

• Added Post Evaluation Stage which summarizes evaluations of all docking submissions sending email to addresses set by --summaryemail flag Issue #111 Issue #110 Issue #100
• If any task/stage fails report that via email to addresses set by --summaryemail flag Issue #125
• Order of evaluation of external dock submissions can now be dictated by a new column in participant_list.csv Issue #124
• A timeout has now been added for blastnfilter stage to prevent the task/stage from running too long. Default is 24 hours, but can be changed with --blastnfiltertimeout flag. Issue #8
• A timeout has now been added for evaluation stage to prevent the task/stage from running too long. Default is 24 hours, but can be changed with --evaluationtimeout flag. Issue #123
• Fixed bug in blastnfilter to correctly rank the list of hiTanimoto candidates.

1.6.8 (2017-03-07)

• Celpprunner puts lock file within specific week directory. Issue #122

1.6.7 (2017-02-07)

• Increased retry time for challengedata download to 1 hour
• Updates to evaluation suffix structure (Allows for more complex suffixes after guids)
• Fixed bugs in CELPPade error messages

1.6.6 (2017-01-23)

• Add retry to external submission downloads. Issue #112

1.6.5 (2017-01-13)

• Fix the bug to skip invalid holo hit instead of terminating the whole blastnfilter process. Issue #89
• Fix the bug in the set sequence fuction where it original complains about the lower cased chain ID

1.6.4 (2017-01-10)

• Remove intermediate files with pymol prefix from challenge data package. Issue #73
• Place docked files in top-level target directory in submissions. Issue #86
• Fixed sporadically failing unit test. Issue #104
• Add evaluation chain permuter implementation and tests. Issue #107

1.6.3 (2016-12-01)

• Modified ParticipantDatabase to optionally strip off _# from guid when doing search for Participant. This is to handle case where single participant has multiple submissions to CELPP. Issue #98
• Switched os.getlogin() calls to pwd.getpwuid(os.getuid())[0] cause os.getlogin() was raising OSError on Travis. Issue #102
• Fixed bug where evaluation task completed email had log messages from other evaluation task emails in them. Issue #99
• Updated readme.txt file in challenge data package to reflect use of new_release_structure_sequence_canonical.tsv instead of new_release_structure_sequence.tsv file. Issue #97

1.6.2 (2016-10-26)

• Fixed bug where large amounts of output to standard out/err caused celpprunner.py to exit due to an exception from smtplib due to very large email. Issue #95

1.6.1 (2016-10-24)

• Added createchallenge stage which is NOT a stage, but a fake stage that runs the following stages: makedb,import,blast,challengedata. Issue #92
• Moved logic to setup logging handlers to start of celpprunner.py to remove no handlers found error for d3r.celpp.util. Issue #91
• Fixed bug where participant_list.csv could not be parsed if file was written with carriage return delimiters instead of newlines. Issue #93
• Cleaned up CELPPade by updating documentation and simplifying variable names
• Version of d3r is now written to 'start' file in each stage/task. Issue #94

1.6.0 (2016-10-13)

• Evaluation task now emails results of evaluation to external submitter. Issues #49 , #81
• Adjusted files uploaded to ftp server in EvaluationTask to reflect changes in output from genchallengedata.py script. Issues #79 , #80
• Added WebDavFileTransfer class to enable upload & download of files via WebDa for celppade tools. Issue #76
• Added tsv files and Components-inchi.ich files to list of files uploaded to ftp by DataImportTask. Issue #78
• Updated challenge data package readme.txt to include documentation for hiTanimoto. Issue #75
• Added pdb_seqres.txt.gz to list of files uploaded to ftp by MakeBlastDBTask. Issue #77
• Download canonical tsv file in data import stage. Issue #84
• Added --rdkitpython flag to celpprunner.py and modified code to pass it to chimera_proteinligprep.py. Issue #88
• Fixed bug where celpprunner would fail if evaluation stage is rerun with completed evaluation tasks. Issue #87
• Improved documentation in RMSD.txt. Issues #82 , #83

1.5.0 (2016-09-11)

• Modified blastnfilter candidate txt file by adding hiTanimoto and adding more information to hiResHolo and SMCSS.
• Genchallengedata.py modified to keep single chains for all holo proteins (LMCSS, SMCSS, hiResHolo, hiTanimoto)
• In blastnfilter, hiResHolo now only has top structure reported and only one chain. Where top structure is highest resolution hit.

1.4.0 (2016-08-11)

• Fixed issue #66 Change candidate category names. Largest is now LMCSS, Smallest is now SMCSS, Apo is now HiResApo, Holo is now HiResHolo

1.3.4

• Fixed issue #58 in chimera_proteinligprep.py code now uses rdkit for 3d conf gen instead of babel

1.3.3 (2016-07-18)

• Fixed issue #60 where challenge data package was NOT being uploaded to remote server

1.3.2 (2016-07-12)

• Removed #8 blastnfilter timeout since it was causing blastnfilter script to hang.
• Blastnfilter.py now uses argparse to parse command line arguments
• Added loggging support into blastnfilter.py

1.3.1 (2016-07-01)

• In proteinligprep.py ligprep command modified. -s 1 -g flags removed and -ns flag added to preserve stereo information

1.3.0 (2016-06-29)

• Fixed bug #45 where autodock vina task was being incorrectly fed proteinligprep as input. Code now feeds it chimeraprep
• #28 Data import stage waits for TSV files to be updated before downloading
• #8 celpprunner will now kill blastnfilter if it runs beyond time set via --blastnfiltertimeout flag
• #37 Added external docking submission task which downloads external docked results so they can be evaluated the same way as the internal docking programs
• #44 Added utility function to call external processes. To reduce redundancy in the code base

1.2.0 (2016-06-03)

• proteinligprep.py and chimera_proteinligprep.py has been updated to work with genchallenge stage output

• vinadocking.py and glidedocking.py now output receptor as pdb and ligand as mol.

• evaluate.py modified to accept new output format as described here:

• ProteinLigPrepTask #41 modified to use ChallengeDataTask as input

• ChimeraProteinLigPrepTask #30 modified to use ChallengeDataTask as input

1.1.0 (2016-05-24)

• ChallengeDataTask now uploads challenge data package (celpp_week##_##.tar.gz) to 'challengedata' directory on ftp if ftpconfig is set properly. This is in addition, to the default upload directory for that challenge data package
• Added a header line in readme.txt of challenge data package to denote start of Blastnfilter summary output.
• Not part of production release, but added prototype vagrant configuration to enable easy creation of a VM that can run celpprunner.

1.0.0 (2016-05-12)

• Added chimeraprep stage to prepare data with Chimera (issue #32)
• Added challengedata stage to generate challenge data package (issue #22) and added genchallengedata.py script which does the work (issue #21)
• Added vina stage to run docking with autodock vina (issue #15) and added vinadocking.py script to run the docking
• Modified D3rTask to write error message into 'error' file (issue #12)
• Added celppreports.py to provide summary reports (issue #14)
• Modified DataImportTask to compare entries in tsv file with data in pdb_seqres.txt in makeblastdb stage. As part of this fix made dataimport stage dependent on makeblastdb stage so the order is now stage.1.makeblastdb => stage.2.dataimport => stage.3.blastnfilter... (issue #16)

0.1.0 (2015-06-30)

• First release on PyPI