<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "JATS-journalpublishing1-3.dtd">
<article article-type="research-article" dtd-version="1.3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xml:lang="ru"><front><journal-meta><journal-id journal-id-type="publisher-id">cheb</journal-id><journal-title-group><journal-title xml:lang="ru">Чебышевский сборник</journal-title><trans-title-group xml:lang="en"><trans-title>Chebyshevskii Sbornik</trans-title></trans-title-group></journal-title-group><issn pub-type="ppub">2226-8383</issn><publisher><publisher-name>Tula State Lev Tolstoy  Pedagogical University</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">10.22405/2226-8383-2018-19-1-187-199</article-id><article-id custom-type="elpub" pub-id-type="custom">cheb-434</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group><subj-group subj-group-type="section-heading" xml:lang="ru"><subject>Статьи</subject></subj-group><subj-group subj-group-type="section-heading" xml:lang="en"><subject>Article</subject></subj-group></article-categories><title-group><article-title>Классификация последовательностей на основе коротких мотивов</article-title><trans-title-group xml:lang="en"><trans-title>Ofitserov Evgeny Petrovich — department of applied mathematics and computer science</trans-title></trans-title-group></title-group><contrib-group><contrib contrib-type="author" corresp="yes"><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Офицеров</surname><given-names>Е. П.</given-names></name><name name-style="western" xml:lang="en"><surname>Ofitserov</surname><given-names>E. P.</given-names></name></name-alternatives><bio xml:lang="ru"><p>Офицеров Евгений Петрович — кафедра прикладной математики и информатики</p></bio><bio xml:lang="en"><p>Ofitserov Evgeny Petrovich — department of applied mathematics and computer science</p></bio><email xlink:type="simple">eofitserov@gmail.com</email><xref ref-type="aff" rid="aff-1"/></contrib></contrib-group><aff-alternatives id="aff-1"><aff xml:lang="ru"><institution>Тульский государственный университет</institution><country>Россия</country></aff><aff xml:lang="en"><institution>Tula State University</institution><country>Russian Federation</country></aff></aff-alternatives><pub-date pub-type="collection"><year>2018</year></pub-date><pub-date pub-type="epub"><day>14</day><month>10</month><year>2018</year></pub-date><volume>19</volume><issue>1</issue><fpage>187</fpage><lpage>199</lpage><permissions><copyright-statement>Copyright &amp;#x00A9; Офицеров Е.П., 2018</copyright-statement><copyright-year>2018</copyright-year><copyright-holder xml:lang="ru">Офицеров Е.П.</copyright-holder><copyright-holder xml:lang="en">Ofitserov E.P.</copyright-holder><license xml:lang="ru" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>Данная работа распространяется под лицензией Creative Commons Attribution 4.0.</license-p></license><license xml:lang="en" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>This work is licensed under a Creative Commons Attribution 4.0 License.</license-p></license></permissions><self-uri xlink:href="https://www.chebsbornik.ru/jour/article/view/434">https://www.chebsbornik.ru/jour/article/view/434</self-uri><abstract><p>Задачи, связанные с классификацией последовательностей символов некоторого алфавита, часто возникают в таких областях, как биоинформатика и обработка естественного языка. Методы глубокого обучения, в особенности модели на основе рекуррентных нейронных сетей, в последние несколько лет зарекомендовали себя как наиболее эффективный способ решения подобных задач. Однако существующие подходы имеют серьезный недостаток — низкую интерпретируемость получаемых результатов. Крайне сложно установить какие именно свойства входной последовательности ответственны за её принадлежность к тому или иному классу. Упрощение же таких моделей с целью повышения их интерпретируемости, в свою очередь, приводит к снижению качества классификации. Такие недостатки ограничивают применение современных методов машинного обучения во многих предметных областях. В настоящей работе мы представляем принципиально новую, интерпретируемую архитектуру нейронных сетей, основанную на поиске набора коротких подпоследовательностей — мотивов, наличие которых влияет на принадлежность последовательности к определенному классу. Ключевой составляющей предлагаемого решения является разработанный нами алгоритм дифференцируемого выравнивания, являющийся дифференцируемым аналогом таких классических способов сравнения строк, как редакционное расстояние Левенштейна и алгоритм Смита–Ватермана. В отличие от предыдущих работ, посвященных классификации последовательностей на основе мотивов, новый метод позволяет не только выполнять поиск в произвольной части строки, но и учитывать возможные вставки.</p></abstract><trans-abstract xml:lang="en"><p>Sequence classification problems often arise in such areas as bioinformatics and natural language processing. In the last few year best results in this field were achieved by the deep learning methods, especially by architectures based on recurrent neural networks (RNN). However, the common problem of such models is a lack of interpretability, i.e., extraction of key features from data that affect the most the model’s decision. Meanwhile, using of less complicated neural network leads to decreasing predictive performance thus limiting usage of state-of-art machine learning methods in many subject areas. In this work we propose a novel interpretable deep learning architecture based on extraction of principal sets of short substrings — sequence motifs. The presence of extracted motif in the input sequence is a marker for a certain class. The key component of proposed solution is differential alignment algorithm developed by us, which provides a smooth analog of classical string comparison methods such as Levenshtein edit distance, and Smith–Waterman local alignment. Unlike previous works devoted to the motif based classification, which used CNN for shift-invariant searching, ours model provide a way to shift and gap invariant extraction of motifs.</p></trans-abstract><kwd-group xml:lang="ru"><kwd>классификация последовательностей</kwd><kwd>машинное обучение</kwd><kwd>нейронные сети</kwd><kwd>поиск мотивов.</kwd></kwd-group><kwd-group xml:lang="en"><kwd>sequence classification</kwd><kwd>machine learning</kwd><kwd>neural network</kwd><kwd>motif extraction</kwd></kwd-group></article-meta></front><back><ref-list><title>References</title><ref id="cit1"><label>1</label><citation-alternatives><mixed-citation xml:lang="ru">Hochreiter S., Schmidhuber J. Long short-term memory // Neural computation. 1997. Vol. 9, № 8. P. 1735–1780.</mixed-citation><mixed-citation xml:lang="en">Hochreiter, S. &amp; Schmidhuber, J. 1997, “Long short-term memory“, Neural computation, vol. 9, no. 8, pp. 1735–1780.</mixed-citation></citation-alternatives></ref><ref id="cit2"><label>2</label><citation-alternatives><mixed-citation xml:lang="ru">Learning phrase representations using RNN encoder-decoder for statistical machine translation / K. Cho [et al.]. // arXiv:1406.1078. 2014.</mixed-citation><mixed-citation xml:lang="en">Cho K. et al. 2014, “Learning phrase representations using RNN encoder-decoder for statistical machine translation“, arXiv:1406.1078.</mixed-citation></citation-alternatives></ref><ref id="cit3"><label>3</label><citation-alternatives><mixed-citation xml:lang="ru">Empirical evaluation of gated recurrent neural networks on sequence modeling / J. Chung [et al.]. // arXiv:1412.3555. 2014.</mixed-citation><mixed-citation xml:lang="en">Chung J. et al. 2014, “Empirical evaluation of gated recurrent neural networks on sequence modeling“, arXiv:1412.3555.</mixed-citation></citation-alternatives></ref><ref id="cit4"><label>4</label><citation-alternatives><mixed-citation xml:lang="ru">Karpathy A., Johnson J., Fei-Fei L. Visualizing and understanding recurrent networks // arXiv:1506.02078. 2015.</mixed-citation><mixed-citation xml:lang="en">Karpathy, A., Johnson, J. &amp; Fei-Fei, L. 2015, “Visualizing and understanding recurrent networks“, arXiv:1506.02078.</mixed-citation></citation-alternatives></ref><ref id="cit5"><label>5</label><citation-alternatives><mixed-citation xml:lang="ru">Lstmvis: A tool for visual analysis of hidden state dynamics in recurrent neural networks / H. Strobelt [et al.]. // IEEE transactions on visualization and computer graphics. 2018. Vol. 24, № 1. P. 667–676.</mixed-citation><mixed-citation xml:lang="en">Strobelt H. et al. 2018, “Lstmvis: A tool for visual analysis of hidden state dynamics in recurrent neural networks“, IEEE transactions on visualization and computer graphics, vol. 24, no. 1, pp. 667–676.</mixed-citation></citation-alternatives></ref><ref id="cit6"><label>6</label><citation-alternatives><mixed-citation xml:lang="ru">Convolutional neural network architectures for predicting DNA–protein binding / H. Zeng et al. // Bioinformatics. 2016. Vol. 32, № 12. P. i121–i127.</mixed-citation><mixed-citation xml:lang="en">Zeng H. et al. 2016, “Convolutional neural network architectures for predicting DNA–protein binding“, Bioinformatics, vol. 32, no. 12, pp. i121–i127.</mixed-citation></citation-alternatives></ref><ref id="cit7"><label>7</label><citation-alternatives><mixed-citation xml:lang="ru">Zhou J., Troyanskaya O.G. Predicting effects of noncoding variants with deep learning–based sequence model // Nature methods. 2015. Vol. 12, № 10. P. 931.</mixed-citation><mixed-citation xml:lang="en">Zhou, J. &amp; Troyanskaya, O.G. 2015, “Predicting effects of noncoding variants with deep learning–based sequence model“, Nature methods, vol. 12, no. 10, pp. 931.</mixed-citation></citation-alternatives></ref><ref id="cit8"><label>8</label><citation-alternatives><mixed-citation xml:lang="ru">Deep motif: Visualizing genomic sequence classifications / J. Lanchantin [et al.]. // arXiv:1605.01133. 2016.</mixed-citation><mixed-citation xml:lang="en">Lanchantin J. et al. 2016, “Deep motif: Visualizing genomic sequence classifications“, arXiv:1605.01133.</mixed-citation></citation-alternatives></ref><ref id="cit9"><label>9</label><citation-alternatives><mixed-citation xml:lang="ru">Quang D., Xie X. DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences // Nucleic acids research. 2016. Vol. 44, № 11. P. e107–e107.</mixed-citation><mixed-citation xml:lang="en">Quang D. &amp; Xie X. 2016, “DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences“, Nucleic acids research, vol. 44, no. 11, pp. e107– e107.</mixed-citation></citation-alternatives></ref><ref id="cit10"><label>10</label><citation-alternatives><mixed-citation xml:lang="ru">Левенштейн В.И. Двоичные коды с исправлением выпадений, вставок и замещений символов // Докл. АН СССР. 1965. Т. 163, № 4. С. 845–848.</mixed-citation><mixed-citation xml:lang="en">Levenshtein, V.I. 1965, “Binary codes with correction of fallouts, inserts and notes“, Reports of the Academy of Sciences (in Russian), vol. 163, no. 4, pp. 845–848. [in Russian]</mixed-citation></citation-alternatives></ref><ref id="cit11"><label>11</label><citation-alternatives><mixed-citation xml:lang="ru">Smith T.F., Waterman M.S. Comparison of biosequences // Advances in applied mathematics. 1981. Vol. 2, № 4. P. 482–489.</mixed-citation><mixed-citation xml:lang="en">Smith T.F. &amp; Waterman M.S. 1981, “Comparison of biosequences“, Advances in applied mathematics, vol. 2, no. 4, pp. 482–489.</mixed-citation></citation-alternatives></ref><ref id="cit12"><label>12</label><citation-alternatives><mixed-citation xml:lang="ru">Gotoh O. An improved algorithm for matching biological sequences // Journal of molecular biology. 1982. Vol. 162, № 3. P. 705–708.</mixed-citation><mixed-citation xml:lang="en">Gotoh, O. 1982, “An improved algorithm for matching biological sequences“, Journal of molecular biology, vol. 162, no. 3, pp. 705–708.</mixed-citation></citation-alternatives></ref><ref id="cit13"><label>13</label><citation-alternatives><mixed-citation xml:lang="ru">Manavski S.A., Valle G. CUDA compatible GPU cards as efficient hardware accelerators for Smith-Waterman sequence alignment // BMC bioinformatics. 2008. Vol. 9, № 2. P. S10.</mixed-citation><mixed-citation xml:lang="en">Manavski S.A., Valle G. 2008, “CUDA compatible GPU cards as efficient hardware accelerators for Smith-Waterman sequence alignment“, BMC bioinformatics, vol. 9, no. 2, pp. S10.</mixed-citation></citation-alternatives></ref><ref id="cit14"><label>14</label><citation-alternatives><mixed-citation xml:lang="ru">Ioffe S., Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift // arXiv:1502.03167. 2015.</mixed-citation><mixed-citation xml:lang="en">Ioffe S. &amp; Szegedy C. 2015, “Batch normalization: Accelerating deep network training by reducing internal covariate shift“, arXiv:1502.03167.</mixed-citation></citation-alternatives></ref><ref id="cit15"><label>15</label><citation-alternatives><mixed-citation xml:lang="ru">Hahnloser R.H.R. et al. Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit // Nature. 2000. Vol. 405, № 6789. P. 947.</mixed-citation><mixed-citation xml:lang="en">Hahnloser R.H.R. et al. 2000, Digital selection and analogue amplification coexist in a cortexinspired silicon circuit“, Nature, vol. 405, no. 6789, pp. 947.</mixed-citation></citation-alternatives></ref></ref-list><fn-group><fn fn-type="conflict"><p>The authors declare that there are no conflicts of interest present.</p></fn></fn-group></back></article>
