ProtHyena Architecture: New Approach to Protein Language Models
Automatically translated
CreatorProtHyenais an innovative model of protein developed by researchers from Tokyo Institute of Technology. This model, protein data Home to analyze Hyena operatorIt is designed as an efficient basic model in terms of fast and welding that uses. Unlike traditional pay-based methods, ProtHyena has the ability to capture single amino acid resolution in both long-range context and actual protein sequences. Researchers model Home They have trained in advance using the data set and made fine adjustment for various protein-related tasks. In this process, in some cases it is comparable to the best approaches available, or even better performance is exhibited.
ProtHyena Architecture
Traditional language models Home Transformer and Home When relying on architectures, the length of the context is limited because these models have the complexity of the quaractic calculation of the attention mechanism, efficiency and processing. ProtHyena is an approach that uses Hyena operator to overcome these limitations. Hyena operatorThe natural protein vocabulary uses the spreader and each amino acid contains special character tokens for filling, separation and unknown characters when addressing as a separate token. Hyena operator consists of a repeating structure, which includes doors on long convocations and element basis. The architecture of ProtHyena offers an extremely effective solution to train protein predictors and leads to a promising way for quick and efficient analysis of biological sequences.
In terms of the capabilities of ProtHyena, it can pass remarkable models in modeling long protein sequences and achieve the latest technology results in certain tasks. This model has shown that it is effective to process long protein sequences and deliver the latest technology performance in various flow down tasks. In particular, contemporary models were left behind in tasks such as Far Homology and Fluorescent prediction. Far HomologyHighest accuracy rate in 31.7% reached, Fluores if it is estimated Tags's value Home 0. 678 as determined. Home Secondary Structure Forecast Home (SSP) and Stability He has shown promising results in his duties.
As a result, ProtHyena is a protein language model that combines Hyena operator to solve the computation challenges facing remarkable models. ProtHyena can efficiently process long protein sequences and offer high performance in various flow down tasks. This is an important progress in protein sequence analysis and can contribute to a deeper understanding of the structures and functions of proteins.
Hyena Operator
Hyena operatorTags ProtHyena It is a technique used in the model and provides some advantages by taking the location of traditional attention mechanisms.
Operators used in language models often perform certain mathematical processes when processing data. Thanks to these processes, it allows the model to understand and analyze data more effectively. For example, attention mechanisms are important parts within the data that the model receives as input "found" provides. In this way, the model focuses on the parts where it finds important, rather than giving equal weight to the whole of the data. Hyena operator puts its own operators in place of their attention mechanisms if used.
Below I will explain the details of how this operator works.
- Protein Embeddin g: Firstly, protein sequences are processed with a layer of embedding, which will make them become numeric vectors.
- ProtHyena Block: This is one of the basic components of the model and includes these steps:
- Add & Norm: Insertion and normalization processes.
- Feed Forward: Advanced feeding network.
- Add & Norm: Another addition and normalization process.
- Hyena Operator: Hyena operator, which is the main innovative component.
- Hyena Operator: This part shows the internal structure of the Hyena operator and consists of the following steps:
- Dense: Intense (Dense) and layered (Convolution).
- Long Convolution: Long range of evasion process.
- Element-wise Gate: Door operation on the basis of the Element, which can be seen as a non-linear activation and a kind of filtering process.
- Dense: Another dense and ripped layer array.
- Hyena Filter: This part defines the filter inside the operator and includes these steps:
- Window, Linear, Sine Act, Linear: The process is done using linear and trigonometric activation functions through a certain window size.
- Location Embed: Positional embedding allows the model to consider the location of every amino acid in the array.
Hyena operator shows how it uses long-range contexts and non-linear activations as an alternative method to its attention mechanism. Hyena operator aims to achieve the same or better performance with less calculation complexity instead of traditional attention mechanisms. This operator allows the model to process long protein sequences more efficiently, which increases overall model performance.
News
- https://www.biorxiv.org/content/10.1101/2024.01.18.576206v1
- https://www.marktechpost.com/2024/01/26/researchers-from-the-tokyo-uranium-of-technology-introduce-prothyena-a-fast-and-efficient-foundation-protein-language-model-at-single-amino-acid-resolution/
- https://cbirt.net/unveiling-prothyena-a-fast-and-efficient-protein-language-model-for-analysis-at-single-amino-acid-resolution/
More
Also for more in the field of artificial intelligence in health ‘Health and Artificial Intelligence: 5 Leading Companies Converting Industry!’ 'See our article.
Creator
