References

  1. Aguet, François, Andrew A. Brown, Stephane E. Castel, Joe R. Davis, Yuan He, Brian Jo, Pejman Mohammadi, et al. 2017. ‘Genetic Effects on Gene Expression across Human Tissues’. Nature 550 (7675): 204–13. https://doi.org/10.1038/nature24277.

  2. Albert, Frank W., and Leonid Kruglyak. 2015. ‘The Role of Regulatory Variation in Complex Traits and Disease’. Nature Reviews Genetics 16 (4): 197–212. https://doi.org/10.1038/nrg3891.

  3. Avsec, Žiga, Vikram Agarwal, Daniel Visentin, Joseph R. Ledsam, Agnieszka Grabska-Barwinska, Kyle R. Taylor, Yannis Assael, John Jumper, Pushmeet Kohli, and David R. Kelley. 2021. ‘Effective Gene Expression Prediction from Sequence by Integrating Long-Range Interactions’. Nature Methods 18 (10): 1196–1203. https://doi.org/10.1038/s41592-021-01252-x.

  4. Avsec, Žiga, Melanie Weilert, Avanti Shrikumar, Sabrina Krueger, Amr Alexandari, Khyati Dalal, Robin Fropf, et al. 2021. ‘Base-Resolution Models of Transcription-Factor Binding Reveal Soft Motif Syntax’. Nature Genetics 53 (3): 354–66. https://doi.org/10.1038/s41588-021-00782-6.

  5. Choromanski, Krzysztof, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, et al. 2021. ‘Rethinking Attention with Performers’. ArXiv:2009.14794 [Cs, Stat], March. http://arxiv.org/abs/2009.14794.

  6. Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. ‘BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding’. ArXiv:1810.04805 [Cs], May. http://arxiv.org/abs/1810.04805.

  7. Edwards, Stacey L., Jonathan Beesley, Juliet D. French, and Alison M. Dunning. 2013. ‘Beyond GWASs: Illuminating the Dark Road from Association to Function’. The American Journal of Human Genetics 93 (5): 779–97. https://doi.org/10.1016/j.ajhg.2013.10.012.

  8. Forrest et al. 2014. ‘A Promoter-Level Mammalian Expression Atlas’. Nature 507 (7493): 462. https://doi.org/10.1038/nature13182.

  9. Gale, Trevor, Matei Zaharia, Cliff Young, and Erich Elsen. 2020. ‘Sparse GPU Kernels for Deep Learning’. ArXiv:2006.10901 [Cs, Stat], August. http://arxiv.org/abs/2006.10901.

  10. Hobert, Oliver. 2008. ‘Gene Regulation by Transcription Factors and MicroRNAs’. Science 319 (5871): 1785–86. https://doi.org/10.1126/science.1151651.

  11. Katharopoulos, Angelos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. 2020. ‘Transformers Are RNNs: Fast Autoregressive Transformers with Linear Attention’. ArXiv:2006.16236 [Cs, Stat], August. http://arxiv.org/abs/2006.16236.

  12. Kelley, David R. 2020. ‘Cross-Species Regulatory Sequence Activity Prediction’. PLOS Computational Biology 16 (7): e1008050. https://doi.org/10.1371/journal.pcbi.1008050.

  13. Kelley, David R., Yakir A. Reshef, Maxwell Bileschi, David Belanger, Cory Y. McLean, and Jasper Snoek. 2018. ‘Sequential Regulatory Activity Prediction across Chromosomes with Convolutional Neural Networks’. Genome Research 28 (5): 739–50. https://doi.org/10.1101/gr.227819.117.

  14. Kelley, David R., Jasper Snoek, and John L. Rinn. 2016. ‘Basset: Learning the Regulatory Code of the Accessible Genome with Deep Convolutional Neural Networks’. Genome Research 26 (7): 990–99. https://doi.org/10.1101/gr.200535.115.

  15. Kitaev, Nikita, Łukasz Kaiser, and Anselm Levskaya. 2020. ‘Reformer: The Efficient Transformer’. ArXiv:2001.04451 [Cs, Stat], February. http://arxiv.org/abs/2001.04451.

  16. Krivega, Ivan, and Ann Dean. 2012. ‘Enhancer and Promoter Interactions — Long Distance Calls’. Current Opinion in Genetics & Development 22 (2): 79. https://doi.org/10.1016/j.gde.2011.11.001.

  17. Leslie, R., C. J. O’Donnell, and A. D. Johnson. 2014. ‘GRASP: Analysis of Genotype-Phenotype Results from 1390 Genome-Wide Association Studies and Corresponding Open Access Database’. Bioinformatics 30 (12): i185–94. https://doi.org/10.1093/bioinformatics/btu273.

  18. Levine, Mike. 2010. ‘Transcriptional Enhancers in Animal Development and Evolution’. Current Biology 20 (17): R754–63. https://doi.org/10.1016/j.cub.2010.06.070.

  19. Long, Hannah K., Sara L. Prescott, and Joanna Wysocka. 2016. ‘Ever-Changing Landscapes: Transcriptional Enhancers in Development and Evolution’. Cell 167 (5): 1170–87. https://doi.org/10.1016/j.cell.2016.09.018.

  20. Mamoshina, Polina, Armando Vieira, Evgeny Putin, and Alex Zhavoronkov. 2016. ‘Applications of Deep Learning in Biomedicine’. Molecular Pharmaceutics 13 (5): 1445–54. https://doi.org/10.1021/acs.molpharmaceut.5b00982.

  21. Pei, Guangsheng, Ruifeng Hu, Yulin Dai, Astrid Marilyn Manuel, Zhongming Zhao, and Peilin Jia. 2021. ‘Predicting Regulatory Variants Using a Dense Epigenomic Mapped CNN Model Elucidated the Molecular Basis of Trait-Tissue Associations’. Nucleic Acids Research 49 (1): 53–66. https://doi.org/10.1093/nar/gkaa1137.

  22. Pennacchio, Len A., Nadav Ahituv, Alan M. Moses, Shyam Prabhakar, Marcelo A. Nobrega, Malak Shoukry, Simon Minovitsky, et al. 2006. ‘In Vivo Enhancer Analysis of Human Conserved Non-Coding Sequences’. Nature 444 (7118): 499–502. https://doi.org/10.1038/nature05295.

  23. Roadmap Epigenomics Consortium, Wouter Kundaje, Jason Ernst, Misha Bilenky, Angela Yen, Alireza Heravi-Moussavi, Pouya Kheradpour, et al. 2015. ‘Integrative Analysis of 111 Reference Human Epigenomes’. Nature 518 (7539): 317–30. https://doi.org/10.1038/nature14248.

  24. Sonnet Developers. 2020. Sonnet Documentation — Sonnet Documentation (version v2.0.0). https://sonnet.readthedocs.io/en/latest/.

  25. Tay, Yi, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler. 2020. ‘Long Range Arena: A Benchmark for Efficient Transformers’. ArXiv:2011.04006 [Cs], November. http://arxiv.org/abs/2011.04006.

  26. TensorFlow Developers. 2022. TensorFlow (version v2.8.2). Zenodo. https://doi.org/10.5281/ZENODO.4724125.

  27. The ENCODE Project Consortium. 2012. ‘An Integrated Encyclopedia of DNA Elements in the Human Genome’. Nature 489 (7414): 57. https://doi.org/10.1038/nature11247.

  28. Uffelmann, Emil, Qin Qin Huang, Nchangwi Syntia Munung, Jantina de Vries, Yukinori Okada, Alicia R. Martin, Hilary C. Martin, Tuuli Lappalainen, and Danielle Posthuma. 2021. ‘Genome-Wide Association Studies’. Nature Reviews Methods Primers 1 (1): 1–21. https://doi.org/10.1038/s43586-021-00056-9.

  29. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. ‘Attention Is All You Need’. ArXiv:1706.03762 [Cs], December. http://arxiv.org/abs/1706.03762.

  30. Woolfe, Adam, Martin Goodson, Debbie K. Goode, Phil Snell, Gayle K. McEwen, Tanya Vavouri, Sarah F. Smith, et al. 2004. ‘Highly Conserved Non-Coding Sequences Are Associated with Vertebrate Development’. PLOS Biology 3 (1): e7. https://doi.org/10.1371/journal.pbio.0030007.

  31. Yao, Zhuliang, Shijie Cao, Wencong Xiao, Chen Zhang, and Lanshun Nie. 2019. ‘Balanced Sparsity for Efficient DNN Inference on GPU’. Proceedings of the AAAI Conference on Artificial Intelligence 33 (July): 5676–83. https://doi.org/10.1609/aaai.v33i01.33015676.

  32. Zaheer, Manzil, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, et al. 2021. ‘Big Bird: Transformers for Longer Sequences’. ArXiv:2007.14062 [Cs, Stat], January. http://arxiv.org/abs/2007.14062.

  33. Zhou, Jian, Chandra L. Theesfeld, Kevin Yao, Kathleen M. Chen, Aaron K. Wong, and Olga G. Troyanskaya. 2018. ‘Deep Learning Sequence-Based Ab Initio Prediction of Variant Effects on Expression and Disease Risk’. Nature Genetics 50 (8): 1171–79. https://doi.org/10.1038/s41588-018-0160-6.

  34. Zhou, Jian, and Olga G. Troyanskaya. 2015. ‘Predicting Effects of Noncoding Variants with Deep Learning–Based Sequence Model’. Nature Methods 12 (10): 931–34. https://doi.org/10.1038/nmeth.3547.