PepPCBench is a Comprehensive Benchmark for Protein-Peptide Complex Structure Prediction with AlphaFold3
1. PepPCBench introduces the most comprehensive benchmark to date for evaluating protein-peptide complex structure prediction using AlphaFold3 (AF3) and other PFNNs, addressing a critical gap in peptide drug discovery tools.
2. At its core is PepPCSet, a rigorously curated dataset of 261 non-redundant protein-peptide complexes (5–30 residues), designed to be independent of any AlphaFold3 training data and structured to reflect real-world docking challenges.
3. The benchmark compares five PFNNs (AF3, AFM, RoseTTAFold-All-Atom, Chai-1, HelixFold3) using standardized metrics like DockQ, L_RMSD, F_nat, MolProbity score, and Rosetta binding energy, offering a multi-dimensional assessment.
4. AF3 consistently outperforms its peers in terms of docking accuracy, structure quality, and ranking power, but still struggles with longer or flexible peptides and real-world virtual screening tasks.
5. The study highlights the structural "memorization" issue in PFNNs, showing that AF3 performs significantly better on peptide complexes similar to its training data, with DockQ dropping from 0.70 (train set) to 0.46 (test set).
6. Peptide length strongly impacts prediction: for peptides under 20 residues, docking accuracy decreases with length due to conformational flexibility; interestingly, AF3 regains accuracy for peptides longer than 20, aligning them more with protein folding behavior.
7. AF3 performs best on “easy” docking cases but still shows solid results on “difficult” complexes, especially when peptides are longer and exhibit protein-like structural constraints.
8. Confidence-related metrics like ipTM and actifpTM correlate well with docking quality, especially for AF3, enabling better model ranking. Combining these with Rosetta binding energy and structural relaxation improves predictive discrimination.
9. Enhanced sampling (increasing seeds and model replicates) has minimal impact on average predictive performance but raises the theoretical success limit, indicating potential for better sampling-aware scoring strategies.
10. AF3’s structural outputs are geometrically and stereochemically robust, with MolProbity and Clash Scores approaching those of native PDB structures—highlighting its utility for downstream applications.
11. A case study on MHC-II peptide binders shows AF3’s structure-based rankings correlate well with experimental affinities. actifpTM proved most reliable for recovering high-affinity peptides, suggesting practical relevance for screening workflows.
12. Despite AF3’s impressive performance, challenges like scoring function optimization and training data bias remain. PepPCBench serves not only as a benchmark but as a practical guide for using PFNNs in peptide-based drug design.
13. The authors emphasize transparency and reproducibility, providing all data, processing protocols, evaluation scripts, and structural models to facilitate adoption and future development.
💻Code:
github.com/zhaisilong/PepPCB…
📜Paper:
biorxiv.org/content/10.1101/…
#peptidedesign #AlphaFold3 #proteinpeptide #structuralbioinformatics #drugdiscovery #molecularmodeling #bioinformatics #PFNN #benchmarking