Let’s say you have some data.

For example, it could be a spreadsheet from your company, a database full of user events, a bunch of medical records, or a TCGA gene expression dataset to make it more specific to life sciences. You want to understand what is inside it, and what insights you can generate from it.

A few years ago, this usually meant choosing one analysis pipeline and then spending a lot of time making it work. You connected the data, cleaned it, transformed it, ran some models, made some plots, and then wrote down what you found. An overwhelming amount of time was actually spent on the first part, that is dubbed data massaging for a reason.

But with AI, the situation becomes a bit stranger, since most of the technical parts are done by it. This opens up the possibility to scale whole thing horizontally.

You no longer have to run only one analysis.
You can run hundreds.

And these analyses do not have to differ only at the end, where the AI writes a different summary. They can differ from the very beginning. One analysis can use different data connectors. Another can clean and join the data in a different way, thus approach the problem from a different angle. Another can use another statistical method. Another can generate a completely different type of insight.

So the question is not simply:

“Can AI analyze this dataset?”

The better question is:

“Which parts of the analysis should be fixed and later make it deterministic and well tested, and which parts should be allowed to change dynamically by the AI to explore the possibilities and find novelty?”

That is the main idea behind I’d like to communicate here to you.

The analysis pipeline recap

For the sake of illustration, imagine that every data analysis has a few layers.

First, there are the connectors. These decide where the data comes from. Maybe we use one database, or five APIs, or uploaded files, or an internal company system.

Then comes the data massaging. This is the ugly but important part. We clean the data, remove bad rows, normalize values, merge tables, filter things, calculate new variables, and make the dataset usable.

After that comes the actual method. This could be a statistical test, a machine learning model, clustering, survival analysis, dimensionality reduction, pathway enrichment, or whatever else fits the question.

Finally, there is insight generation. This is where the results become something a person can use. A report, a recommendation, a ranking, a warning, a hypothesis, a figure, or a list of next steps.

The interesting part is that each layer can be fixed or flexible. We don’t want something that works to be flexible enough to risk that the LLM will hallucinate something.

You can fix the connectors but allow the methods to vary.

You can fix the data cleaning but allow the AI to generate many different interpretations.

You can fix the final report format but allow the system to search through different data sources.

Or you can let the whole thing vary from beginning to end.

That last version is the most open-ended. It is also the hardest to control.

A simple example if you are looking for an apartment

Imagine you want to choose a new apartment. You have some basic data: rent, size, location, commute time, noise level, safety, sunlight, and energy costs.

A normal analysis might rank the apartments with a fixed scoring system. Rent gets some weight. Commute time gets some weight. Size gets some weight. The apartment with the highest score wins.

This is already useful but it is only one possible analysis.

An AI system could do something more flexible. It could connect to rental websites, public transport data, map data, neighborhood statistics, noise maps, and maybe even weather or sunlight information. It could also process the same data in different ways.

One version calculates rent per square meter. Another estimates the real commute during rush hour. Another removes all apartments where the total monthly cost goes above your limit. Another tries to estimate which apartment you will still like after six months.

Then the method can change too.

One analysis finds the cheapest acceptable apartment. Another finds the best balance between price and comfort. Another groups the apartments into types, such as cheap, convenient, quiet, or risky. Another tries to find the apartment that is least likely to become annoying later.

The output can also change.

One report might say:

“Choose Apartment A. It is the cheapest option that still meets your basic requirements.”

Another might say:

“Choose Apartment B. It is not the cheapest, but it saves you the most time every week.”

The format of the output can also vary, one might prefer a pdf report, another person prefer in a HTML format.

All of these can come from the same starting data.

So what should we do?

Maybe we decide that the data sources should be fixed. We trust only a few apartment websites, but we allow the AI to try many scoring systems.

Or maybe we do the opposite. We allow the AI to collect data from many places, but we force it to use our fixed personal scoring model.

Or maybe we fix the final output. Every time, the system must give us three answers: cheapest, most balanced, and best long-term option.

This is already very different from simply asking an AI to “analyze the data.”

We are designing which parts of the analysis are allowed to move, and which ones to hardcode and make reliable.

An example from cancer research

Suppose we want to analyze TCGA gene expression data. We want to find genes, pathways, or patterns that are related to survival, cancer subtype, or disease biology. This sounds like one task, but it is not.

At the connector level, we have many choices. We can use only RNA-seq expression data. Or we can add clinical survival data. Or mutation data or any other modalities. The input can even be derived raw results from another machine learning algorithm.

Already, the analysis can branch in many directions, depending on what hypothesis we have.

Then comes preprocessing.

Do we use raw counts, normalized counts, TPM, or FPKM? Do we log-transform the values? Do we remove low-expression genes? Do we correct for batch effects? Do we adjust for tumor purity? Do we split the cohort by subtype? What do we do with missing clinical variables? Each of these choices changes the result.

Then comes the method part.

We could run differential expression analysis. We could run univariate Cox regression. We could run multivariate Cox regression adjusted for age, stage, and tumor purity. We could make Kaplan-Meier groups. We could cluster samples. We could run pathway enrichment. We could build a classifier. We could look for gene modules instead of single genes.

And after all of this, the AI still has to decide what kind of insight to produce.

Maybe it writes a biomarker discovery report, or pathway-level interpretation or maybe it produces a clinical risk hypothesis. All of these analyses can begin with the same TCGA cohort. But they are not the same analysis.

What should be fixed and where should we allow the AI to be creative?

This is where the design choice becomes important.

If we allow the AI to change everything, then it can explore a huge space of possible analyses. This can be useful for discovery. Maybe it finds a signal that we would not have checked manually.

But the result can become messy if every analysis uses different connectors, different preprocessing, different methods, and different insight formats, then comparing the outputs becomes difficult.

So we can freeze part of the pipeline.

For example, we can fix preprocessing:

“Always use the same RNA-seq normalization, filtering, and clinical covariate handling. After that, let the AI explore different biological interpretations.”

This gives us stable data handling but flexible insight generation. The important part is that this “fixing” can mean that we actually write a deterministic code to do that part of the analysis, or we use LLM agents, but strictly define the constraints in what it can do.

We can also fix the final insight format:

“Always produce a survival-biomarker report. But allow the AI to test expression, mutation, methylation, immune infiltration, and external validation data.”

This keeps the report comparable while allowing the data sources to expand.

Or we can fix the method:

“Always use multivariate Cox regression adjusted for age, stage, and tumor purity. But allow different feature-selection and preprocessing strategies.”

This makes the statistical comparison cleaner.

Why this matters

Most people talk about AI data analysis as if the main benefit is speed, but that is only part of it.
The more interesting benefit is that AI lets us explore many possible analytical paths at once, and let’s us test many many different hypotheses.
But this creates a new problem. If everything can vary, then we need to decide what should not vary, since we actually want to understand what we are doing.

In some cases, we want discovery. Then we should allow more flexibility.

In other cases, we want reproducibility. Then we should fix more of the pipeline.

Sometimes we want to test robustness. Then we keep one part fixed and vary another part on purpose.

The important part is that:

You can outsource the technical implementation to the AI, but you cannot outsource your understanding.

And to help with understanding, you need to know that what part of the data analysis you can trust, and which part can introduce LLM errors.

What is vibe coding?

If I could have, I would have picked “vibe coding” as the word of the year in 2025, or at the very least in 2026. So what is it?

You have an idea. For example, it would be nice to automatically organize your emails, or build a new app for your phone, or automate something at work that until now could only be done through manual clicking. A few years ago, even getting started on something like this required years of experience. But in 2026, it looks more like this:

You crack open a white Monster:

You put on a pair of glasses like this:

And you download a so-called coding agent, for example Codex, which comes with a ChatGPT subscription, Claude Code, or Devin. We will be using the latter, because it includes access to free models.

The idea behind vibe coding is very simple: you do not program by writing every single line of code yourself. Instead, you tell the AI what you want to build, then you shape the application together while you “vibe”, meaning you enjoy the simple joy of creating something. Or you fight off a headache when the AI writes the code badly. Below, I will show you the easiest way to get started with this technique, but first a warning:

Vibe coding is a cool way to start creating apps and projects if you are not familiar with the technical details and you have ideas that won’t cause a problem if they go sideways. But please, consult a software engineer for any serious projects, where for example, security matters. You want to do some scientific work? Ask a (human) scientist first!

Your first vibe-coded app

For the sake of illustration, I chose a simple problem that shows how vibe coding works. I have always been curious about the different kinds of things people dream about, and which themes come up again and again for many people. To explore this, we will build a multiplayer online web app where users can submit written descriptions of their dreams. These dreams will then automatically cluster into an interesting, map-like visualization. Players who submit the strangest dreams, the ones that differ most from everyone else’s, will appear on a leaderboard.

1. Download and install Devin

Devin is an application that lets you interactively talk to an AI-based developer. As I mentioned earlier, we will be using Devin to make the basics of vibe coding easy to demonstrate, but everything you see here also applies to tools like Codex.

You can download Devin here:
https://devin.ai/download

Register here:
https://app.devin.ai/login?prompt=empty

Once you open Devin on your computer, you can sign in and connect it to your account.

2. Getting familiar with the Devin interface

First, create a folder on your computer, for example named vibe_coding_demo. After opening that folder, the interface will look like this:

For now, the part we care about is in the bottom right. This is where we can choose the AI coding models. If you are paying for Devin, then at the time of writing I recommend the Claude Opus 4.6 Thinking model. It gives the best performance, although unfortunately its usage limit runs out the fastest. If you are using the free version, Kimi K2.6 is usually quite good for simpler projects.

I will not go deeper into the interface for now. I will cover that in the next demos.

Next, in the chat window on the right, enter this prompt:

“Create a goal prompt for the following: I have always been curious about the different kinds of things people dream about, and which themes come up again and again for many people. To explore this, we will build a multiplayer online web app where users can submit written descriptions of their dreams. These dreams will then automatically cluster into an interesting, map-like visualization. Players who submit the strangest dreams, the ones that differ most from everyone else’s, will appear on a leaderboard.”

After that, we can write:

“yes, start building the application, test it, and do not stop until it works completely.”

Then we can choose whether to go get a coffee, do push-ups, or whatever else we feel like. The full process can take hours, and if the AI needs our help, it will let us know. It is worth keeping the terminal open, because sometimes we need to approve things during installations.

As the project is being developed, we can see the code appearing in the folder, and it is also visible through Devin. We can accept the changes manually, although this can also happen automatically.

After about 20 minutes, it is ready:

Opening the website and adding a new dream:

We can see that the program placed the dream in the right area, close to “Flying and heights”.

Summary

This article was only a very simple example of how you can start vibe coding. From here, I plan to return week by week with more complex topics and features inside Devin.

AI systems are becoming much better at scientific reasoning. They can answer difficult technical questions, write code, summarize papers, use tools, and combine information across domains. In some benchmark settings, their performance now looks close to, or above, expert human performance.

This creates an obvious question for biology: if AI can reason so well, where are the discoveries?

In other words, if AI is so smart, why can’t I just do this:

And get meaningful answers to solve new scientific problems.

The answer is not that biology has been ignored by AI, but quite the opposite. Foundation models are now being trained on genomes, proteins, single-cell data, multimodal biological datasets, and the scientific literature itself. Models can classify DNA sequences, predict protein properties, annotate cells, prioritize genes, search papers, write analysis code, and generate biological hypotheses.

Still, a gap remains between prediction and discovery. A model can predict that a genomic region looks regulatory, that a protein mutation may matter, or that a gene appears in the literature around a disease. But those outputs are not automatically scientific findings and they need to be connected to mechanism, disease context, experimental evidence, and eventually to something that can be tested.

The central issue is that biology does not have one language. It has several. We are used to ask questions to language models like chatgpt that answers is in the same language, but some questions can’t be stated in human language.

To take the example from the last figure, if we would just naively copy-paste nucleotide sequences to chatgpt, we would miss a lot of context for a gene, simply because the model does not have the capacity to look at the gene’s neighbourhood:

DNA has a language of nucleotides. Proteins have a language of amino acids. Cells have patterns of gene expression that can be treated, imperfectly but usefully, as a kind of biological sentence. Scientific papers and databases have human language.

Foundation models are becoming useful because they can operate on these different languages. The hard part is understanding what each level can and cannot tell us.

Human language and biological language

A human sentence is more than a string of words. If we read “It is a beautiful sunny day in Boston,” we understand syntax, meaning, location, weather, sentiment, and background knowledge. We know that Boston is a city, that sunny weather is usually positive in this context, and that the sentence describes a situation. This is why it’s quite impressive that a model, like a Large Language Model can implicitly learn all these features by just doing next token prediction.

DNA and protein sequences also look like strings, but their meaning comes from biological context. A DNA motif can matter because it affects regulation. A regulatory element can matter because it changes gene expression. A protein sequence can matter because it folds into a structure. A structure can matter because it changes binding, stability, or function. A gene expression pattern can matter because it reflects a cell state or a disease process.

The analogy between biology and language is useful, as long as it is not taken literally. DNA is not English, a gene is not a word in the normal linguistic sense, and a cell is not simply a sentence. But the analogy helps clarify what foundation models are trying to do. They learn statistical structure from large biological corpora, then reuse that structure for downstream tasks.

The question is what kind of structure they learn, and whether that structure is enough to support interpretation.

Level 1: nucleotide sequence models

The first level is DNA and RNA sequence. These models operate directly on strings of A, T, G, and C. They are trained on large collections of genomic sequence and then adapted to tasks such as promoter prediction, enhancer prediction, variant effect prediction, sequence classification, clustering, anomaly detection, or sequence generation.

Models such as DNABERT2, HyenaDNA, and Evo 2 represent this direction. DNABERT2 was designed for easier installation and fine-tuning on DNA sequence tasks. HyenaDNA focused on long-range genomic modeling at single-nucleotide resolution. Evo 2 extends this idea further by training on trillions of DNA bases and handling very long sequences.

The reason long context matters is that genome function is not always local. A promoter-proximal region may sit close to a transcription start site, but enhancers, silencers, insulators, and other distal regulatory elements can be far away. A relevant regulatory neighborhood may span tens of thousands, hundreds of thousands, or even millions of bases.

This is a major difference from ordinary text. In human language, nearby words often matter most. In genomes, the important element may not be nearby in the linear sequence, and the three-dimensional organization of the genome can matter as much as the local letters.

Nucleotide models can therefore provide powerful representations of sequence, but they usually do not produce a complete biological explanation. They may tell us that a region looks functional or that a variant may have an effect. The next question is what that effect means in a specific biological system.

Level 2: amino acid sequence models

The second level is protein sequence. Instead of four nucleotides, protein models operate on around twenty amino acids. This level is closely connected to function because protein sequence influences structure, and structure influences biological activity.

A protein is written as a one-dimensional sequence, but it acts as a three-dimensional object. Its fold determines binding surfaces, stability, enzymatic activity, interaction partners, and sometimes disease relevance. This makes protein language models useful for tasks such as protein classification, structure-related prediction, mutation effect estimation, functional annotation, and de novo protein design.

Common model families include ESM, ProtTrans models such as ProtBERT and ProtT5, ProteinBERT, generative models such as ProGen and ProtGPT2, and evolution-aware models such as MSA Transformer. These models differ in architecture and training strategy, but they share the idea that large-scale sequence learning can produce reusable protein representations.

Protein models often feel closer to mechanism than DNA models because the path from sequence to function is more direct. A mutation can change an amino acid, the amino acid change can alter structure, and that structural change can affect binding or activity.

Level 3: hybrid biological models

The third level is hybrid modeling, where the goal is to connect multiple biological representations, or define even new representations for a given problem. This is important because real biology does not live in one data type. DNA, RNA, protein, chromatin, methylation, mutation status, copy number, cell identity, tissue structure, and clinical outcome are different views of the same underlying system.

Single-cell foundation models are one example. In single-cell genomics, a common analogy is that genes are words and cells are sentences. The analogy is not perfect, but it captures something important: a cell state is not defined by one marker gene. It is defined by a pattern across many genes.

Models such as scGPT use this kind of idea to learn from single-cell genomics data. They can support cell-type annotation, gene network inference, perturbation prediction, and other tasks where the relationships between genes and cells matter.

Other hybrid systems connect protein representations with human language. ProteinChat, for example, points toward an interface where protein structure and function can be explored through natural language. This is not just a convenience feature, but it is part of a larger shift where models begin to translate between biological representations and human interpretation.

Level 4: human scientific text

The fourth level is human text. Biology has produced an enormous written record: papers, abstracts, gene descriptions, pathway databases, clinical annotations, methods sections, reviews, and supplementary tables. LLMs are strong at this level because they can search, summarize, compare, and reorganize this information at a scale that is difficult for humans, but also the end results is understandable by us.

This creates practical applications. LLMs can help with gene prioritization, disease-gene association mining, microbiome-disease relationship discovery, cancer predisposition gene identification, literature screening, and interpretation of pathway results. They can take a long list of genes and produce a structured summary of known functions, possible mechanisms, disease associations, and supporting evidence.

But this level has a major limitation: the literature is not biology itself. It is what humans have already written about biology.

That means LLMs inherit the biases of the scientific record. Some genes and pathways are heavily studied, while others have little literature. Some organisms, diseases, and protein families are overrepresented in training data and databases. Negative results are often missing. Old assumptions can be repeated until they look more certain than they are.

A text-based LLM can therefore produce a convincing explanation that mostly reflects the field’s existing attention patterns.

Level 5: LLMs for scientific data analysis

The fifth level is scientific data analysis itself. Here, the LLM is not only reading a sequence, a protein, a cell profile, or a paper, but it is writing code, choosing analysis methods, generating plots, comparing outputs, calling tools, and producing reports.

This is probably one of the most practically useful roles for LLMs in biology, because a lot of scientific work involves technical implementation. Data has to be cleaned, normalized, transformed, joined, filtered, visualized, modeled, and checked. An LLM can speed up many of these steps.

An LLM can help implement these steps, but it can also make hidden mistakes. This is why data-analysis agents need constraints. Some parts of the workflow should be flexible, especially during exploration. Other parts should be fixed, tested, and reproducible.

High-throughput LLM screening

One of the more interesting directions is high-throughput LLM screening. Instead of asking an LLM to interpret one gene, pathway, or result, the system can screen many candidates and rank them according to biological relevance, literature support, novelty, mechanistic plausibility, or disease connection.

This is useful because modern biological datasets often produce too many candidates for manual interpretation. A differential expression analysis, survival screen, perturbation experiment, or model interpretation step may return hundreds or thousands of genes. An LLM can help triage this space by turning raw candidates into a smaller set of hypotheses.

However, this approach is only useful if the screening is grounded and controlled. If the LLM simply writes plausible stories for every candidate, the result is not much better than automated biological storytelling. The useful version has to separate evidence from speculation, known biology from novelty, and literature frequency from actual mechanistic relevance.

This is where LLMs may become better than random guessing or simple non-LLM ranking methods, but still not useful in their raw form. The output has to be connected back to data, validation, and experimental design.

If you are interested in this research, we have actually sent in a manuscript, where you can read about it on more details: https://www.biorxiv.org/content/10.64898/2026.01.28.702211v1.article-metrics

Limitations

Foundation models in biology have several limitations. Interpretability remains difficult. A model can make a correct prediction without giving a clear reason that a scientist can inspect. Bias in training data is also a serious issue, since biological databases such as UniProt and large genomic resources are not neutral samples of life. They reflect what has been measured, curated, funded, and published.

There is also the problem of catastrophic forgetting. A pretrained model may perform well in general, then lose useful capabilities when fine-tuned on a narrower task. Compute cost is another limitation, especially for models that need long genomic context or large-scale multimodal integration.

But the deeper limitation is conceptual. Different models produce different kinds of outputs, and those outputs should not be treated as equivalent. A DNA model may be right about sequence pattern. A protein model may be right about structural similarity. A single-cell model may be right about cell-state relationships. A literature model may be right about what has already been published. A data-analysis agent may be right about a statistical association.

A scientific claim often needs several of these forms of evidence to line up.

Conclusion

Frontier AI models are already routinely used to solve difficult problems in fast-to-validate fields like mathematics. Real breakthroughs in fields like biology is still to be seen.

However, foundation models are already changing how biologists work, but their real value will not come from treating their outputs as discoveries on their own. Instead, their strength lies in helping scientists connect biological languages across DNA, proteins, cells, literature, and data analysis, while keeping each prediction grounded in mechanism and evidence.

The future of AI in biology will therefore depend less on asking models for final answers, and more on using them to generate clearer, testable hypotheses that can be validated experimentally.

If you’d like to read more articles like this:

Subscribe now

Share

You do (or don’t) agree with something? Leave a comment:

Leave a comment

Disclaimer

_{The article above was presented at the AI Health Frontiers 2026 conference in Boston (https://luma.com/hlegc5mo), and partly showcases PharosBio-s capabilities (www.pharos.bio). You can try out the scientific discovery data analysis here: https://pharos.bio/join?type=waitlist}

The breakthroughs in the field of artificial intelligence in recent years were spearheaded by the so-called Large Language Models (LLMs), including ChatGPT. Their common feature is that they are trained on an incredible amount of online data (in the case of ChatGPT, practically a significant part of the whole internet).

To simplify it, imagine showing the raw model all the Wikipedia articles, all the digitized books, many forum posts, etc., and then covering certain words in these documents and asking the model to tell us what the covered word was based on the context. For example, if the model gets the following sentence: "I got tired of writing the article, I'm going down to grab a __," the correct word is likely to be "coffee" (or "beer") from the text context.

That's it, essentially, there's nothing more complicated happening during the training of LLMs. The problem arises that the resulting system is a massive untamed alien intelligence that can very convincingly help us in our work, but will equally energetically write toxic essays on any topic. As a result, we want to steer this trained model in a direction that aligns with our values, so we basically talk to real people using a method called RLHF until we can put a human mask on this monster.

After that, it's up to us what kind of personality we want to endow this somewhat friendlier system with, and that's where the prompts come in. These are essentially texts that can tell the model how to behave and what kind of output to produce.

To help with this, I can recommend the following list, which is available here: https://github.com/f/awesome-chatgpt-prompts

So, for example, if we want to cook something delicious but have no ideas, we can give the following prompt:

“I need someone who can suggest delicious recipes that provide nutrient-rich dishes, are easily prepared and don't require a lot of time, making them suitable for busy people, among other factors such as cost-effectiveness, so the result is healthy and economical at the same time. The first request: "Something light that can be quickly prepared during the afternoon break".

With this, we basically give the system the necessary information:

• The personality to simulate (the context will show if it's a chef or cookbook, but we can also write it down)

• The task (suggesting recipes)

• A more detailed expansion of the task, limitations

• Output format (it will know to return a recipe with a fixed format, but we can also ask for an excel sheet sorted data set, or write a poem from the result, etc.)

• Try the rest too, especially the language teacher or programming helper can be very useful.

For those who want to learn more about the above hand waving introduction, I can suggest these articles:

Good technical introduction for LLMs: https://jalammar.github.io/illustrated-transformer/

Details on how RLHF works: https://huggingface.co/blog/rlhf

Other prompt ideas: https://mpost.io/100-best-chatgpt-prompts-to-unleash-ais-potential/

And a recent community project has started where they aim to fine-tune a ChatGPT-like system with community help (which will likely better represent the diverse ideologies in the world), anyone can join and help:

https://open-assistant.io/hu/dashboard

2. Talk to your documents

Yes, literally. Essentially, for example in the case of ChatGPT, about 8000 words is what the model can comprehend at once, which we call the context window. This limits what the system will remember during the conversation. As a result, if we talk to it a lot, it will forget what was discussed at the beginning of the discourse.

Fortunately, with the help of the Large Language Models, it is also easy to embed any length of documents into the system with the help of some representative indexing, and then use this representation to initiate a conversation about the given document. This is particularly interesting because, as we saw in the first part, these models were never directly trained for such tasks, but the simplicity of understanding text and retrieval emerges as an emergent property (I will write about this in another post).

Let's look at an example! I found a Hungarian document about the 1848 Hungarian Revolution and War of Independence in a pdf document (http://misc.bibl.u-szeged.hu/14283/1/honved_003_002_003_014-017.pdf), and then generated an embedding and indexing for it using this tool: https://huggingface.co/spaces/rsunner/GPT-Index_simple_upload, which outputs a json file. This file can then be read by the GPT-Index system (available here: https://github.com/jerryjliu/gpt_index) and we can ask questions such as, what was the 12 points demanded by the revolutionists (sorry for my English readers for the Hungarian document, you have to believe me that the results are correct on this one :) Also, Hungarian is a difficult language, so it is a good test for the model.):

Okay, the order got a bit mixed up, but the essence can be seen. We can ask even more complex questions and it can be imagined how many places this system will be useful. For example, the entire Bible has already been embedded/indexed using this method, but lawyers may start to worry if enough legal documents can be stored in this way.

The above methods may seem a bit crude, but if someone is interested in the topic in more depth, they can chat with any uploaded document within professional frameworks for a monthly fee of $5-10 at the following websites:

https://www.usefini.com/

https://www.filechat.io/

Subscribe now

Share

Leave a comment

Az elmúlt évek nagy áttöréseit a mesterséges intelligencia területén az úgynevezett Nagy Nyelvi Modellek (Large Language Models, LLM) jelentették, ide tartozik a ChatGPT is. Ezek közös jellemzője, hogy irgalmatlan mennyiségű online elérhető adaton (ChatGPT esetében gyakorlatilag az egész internet egy jelentős részén) tanítják őket. Ezt leegyszerűsítve úgy kell elképzelni, hogy megmutatjuk a nyers modellnek az összes Wikipedia cikket, az összes digitalizált könyvet, rengeteg fórum posztot, stb, majd ezekben a dokumentumokban kitakarunk bizonyos szavakat, és a szavak környezetéből kérjük meg a modellt, hogy mondja meg, mi volt a kitakart szó. Például, ha a következő mondatot kapja a modell: “Elfáradtam a cikk írása közben, leugrom inni egy ____“, akkor a helyes szó nagy valószínűséggel a “kávét“ lesz a szövegkörnyezetből (vagy “sört”).

Ennyi, lényegében semmi más bonyolultabb dolog nem történik az LLM-ek tanítása közben. A probléma ott jelentkezik, hogy az így kapott rendszer egy hatalmas megzabolázatlan idegen intelligencia, ami majd nagyon meggyőzően tud segíteni a munkánk során, de ugyanolyan lelkesen ír majd toxikus jellegű esszéket bármilyen témában. Ebből kifolyólag szeretnénk ezt a betanított modellt valahogy a mi értékrendűnkel megegyező irányba terelni, így egy RLHF-nek nevezett módszerrel lényegében addig beszéltetjük valódi emberekkel, amíg egy emberi maszkot tudunk rakni erre a szörnyre.

Ezek után már rajtunk áll, hogy milyen személyiséggel szeretnénk felruházni ezt a kicsit barátságosabbá tett rendszert, és itt jönnek képbe a prompt-ok. Ezek lényegében olyan szövegek, amikkel meg tudjuk súgni a modellnek, hogy hogyan viselkedjen és milyen kimenetet produkáljon.

Ehhez segítségül szolgálhat a következő lista: https://github.com/f/awesome-chatgpt-prompts

Tehát például, ha szeretnénk valami finomat főzni, de nincs ötletünk, akkor megadhatjuk a következő promptot:

Szükségem van valakire, aki olyan finom recepteket tud javasolni, amelyek tápanyagokban gazdag ételeket szolgáltatnak, könnyen elkészíthetők és nem igényelnek sok időt, így megfelelőek az elfoglalt emberek számára, más tényezők mellett, mint például a költséghatékonyság, így a végeredmény egyszerre egészséges és gazdaságos. Az első kérés: "Valami könnyű, ami gyorsan elkészíthető a délutáni szünet alatt”.

Ezzel lényegében megadjuk a rendszernek a szükséges információkat:

1. Milyen személyiséget szimuláljon (itt a szövegkörnyezetből tudni fogja, hogy egy szakácsot, vagy szakácskönyvet, de ezt külön is odaírhatjuk)

2. Mi a feladat (recepteket javasolni)

3. A feladat részletesebb kibontása, korlátai

4. Kimenet formátuma (itt tudja, hogy receptet adjon vissza, aminek elég kötött a formája, de akár kérhetjük, hogy egy excel-táblába rendezett adatsort adjon, vagy írjon egy verset az eredményből, stb)

Próbáljátok ki a többit is, különösen hasznos lehet még például a nyelvi tanár, vagy programozói segéd is.

Aki szeretne többet tudni a fenti kézlengetős bevezetésen kívül ezeket a cikkeket tudom javasolni:

LLM-ekhez jó technikai bevezető: https://jalammar.github.io/illustrated-transformer/

Hogyan működik az RLHF részleteiben: https://huggingface.co/blog/rlhf

Egyéb prompt ötletek: https://mpost.io/100-best-chatgpt-prompts-to-unleash-ais-potential/

Illetve nemrég indult egy közösségi projekt, ahol egy ChatGPT-hez hasonló rendszert kívánnak közösségi segítséggel finomhangolni (ami így majd feltehetően jobban reprezentálja a sokfajta ideológiát a világon), bárki bekapcsolódhat és segíthet:

https://open-assistant.io/hu/dashboard

2. Beszélgess a dokumentumaiddal

Igen, szó szerint. Alapvetően például a ChatGPT esetében nagyjából 8000 szó az, amit a modell át tud látni, ezt hívjuk kontextus ablaknak. Ez korlátozza, hogy a beszélgetés során például mire emlékszik majd a rendszer. Ebből következik, hogy ha sokat beszélgetünk vele, el fogja felejteni miről volt szó a diskurzus elején.

Szerencsére a Nagy Nyelvi Modellek segítségével arra is könnyen van lehetőség, hogy tetszőleges hosszúságú dokumentumokat ágyazzunk be a rendszer segítségével, valamilyen reprezentatív indexelés segítségével, majd ezt a reprezentációt használjuk arra, hogy beszélgetést kezdeményezzünk az adott dokumentumról. Ez különösen azért érdekes, mert ahogy az első részben láttuk, ezek a modellek ilyen feladatokra soha nem voltak közvetlenül tanítva, a szöveg megértése és az előhívás egyszerűsége mint emergens tulajdonság jelenik meg (erről majd egy másik posztban írok).

Nézzünk egy példát! Találtam egy magyar dokumentumot az 1848-as magyar forradalom- és szabadságharcról egy pdf dokumentumban (http://misc.bibl.u-szeged.hu/14283/1/honved_003_002_003_014-017.pdf), majd ezzel az eszközzel: https://huggingface.co/spaces/rsunner/GPT-Index_simple_upload generáltam hozzá egy beágyazást és indexelést, aminek a kimenete egy úgynevezett json fájl. Ezt a fájlt utána már beolvastathatjuk a GPT-Index nevű rendszerrel (elérhető itt: https://github.com/jerryjliu/gpt_index) és feltehetünk kérdéseket, mint például, hogy mi volt a 12 pont:

Rendben, a sorrendet kissé összekeverte, de a lényeg látszik. Ennél bonyolultabb összefüggésekre is rákérdezhetünk, és el lehet képzelni mennyi helyen lesz majd hasznos ez a rendszer. Példának okáért a teljes Bibliát beágyazták/beindexelték már ezzel a módszerrel, de az ügyvédek is elkezdhetnek majd izgulni, ha elegendő mennyiségű jogi dokumentumot lehet vele majd ilyen módon tárolni.

A fenti módszerek egy kicsit fapadosnak tűnhetnek, de ha valakit bővebben érdekel a téma, akkor az alábbi weboldalakon 5-10 dollár havidíjért cserébe profibb keretek közt lehet beszélgetni bármilyen feltöltött dokumentummal:

https://www.usefini.com/

https://www.filechat.io/

Subscribe now

Share

Leave a comment

1. Character.AI

Nem szigorúan véve újdonságról van szó, de most hogy a ChatGPT már rendesen beszivárgott a köztudatba, érdemes megemlíteni a Character.AI nevű zseniális projektet, ami már több mint fél éve fejlesztés alatt áll. Lényegében egy ChatGPT-hez hasonló modell áll mögötte, tehát alapvetően egy chatbotról van szó, amit viszont akár mi is pár mondatnyi leírás után és egy próbabeszélgetés során saját személyiséggel ruházhatunk fel.

Az olyan egyszerű alkalmazásokon túl, mint hogy Einstein segítsen megoldani a fizika házid, vagy Elon Musk adjon befektetési tanácsokat, hamar el lehet jutni a kreatívabb chatbotokig, mint például a “WhoWouldWin“ nevű előre elkészített személyiség, aki bármilyen két dologról megmondja, hogy melyikük győzne, utána pedig leállhatsz vele vitatkozni, ha nem értesz egyet. Tudom ajánlani a cipőtalpamra ragadt rágó vs 67 liter Starbucks kávé és hasonló összehasonlításokat, elég jó. Vagy egyik nap volt egy mély beszélgetésem egy székkel (!), majd miután összevesztünk valamin, szó szót követett és hát szerencsétlennek kitört az egyik falába (ő ütött először) és egy percekig tartó lélekbe markoló monológgal jutalmazott az ember-szék alá-fölé rendeltségi viszonyokról mire végül kibékültünk. Igen, valószínűleg én leszek az első a Boston Dynamicsos robotrugdosó mérnökök után aki majd az AI bíróság elé fog állni a közelgő AI apokalipszis után (aki nem ismerné, itt van, de csak erős idegzetűeknek:

).

Külön zseniális opció, amikor egy chatszobába össze lehet ereszteni tetszőleges számú chatbot személyiséget, akik elkezdenek egymással beszélgetni. Az előbb említett összes botot beraktam például egy szobába és magukra hagytam 10 percre és a legabszurdabb bizarr dráma kerekedett ki belőle.

Szóval tudom ajánlani kipróbálásra az oldalt, folyamatosan fejlesztik, de egyelőre teljesen ingyenes. Ha készítetek ti is valami érdekes AI személyiséget hozzá, írjátok meg a poszt alá kommentben, kíváncsi vagyok. (https://beta.character.ai/)

2. Nothing, Forever

Képzeljünk el egy TV műsort, vagy sorozatot, amiben nincsenek előre megírt dialógusok, se előre megírt cselekmény, hanem az egészet egy mesterséges intelligencia generálja valós időben és a történet organikusan változik mindeközben. A műsorba bármikor bekapcsolódhatunk, mert 24/7 alakul az eseménysor. Ahogy egyre jobban teret nyernek és egyre gyorsabbak lesznek az előbb bemutatott szöveggeneráló modellek, úgy csak idő kérdése volt, hogy mikor kezdenek el a fejlesztők kísérletezni ezzel a koncepcióval és az első ilyen próbálkozás a Twitch-en futó Nothing, Forever.

Lényegében a Seinfield sitcom sorozaton alapuló, ChatGPT-szerű AI-val meghajtott végtelenített stream-ről van szó, ahol a szöveg, a kötelező nevetősáv, a cselekmény és a kameramozgások is AI által generáltak. Rendben, a kinézete egyelőre borzasztó és a régi PS1-es grafikát idézi, de van valami bizsergetően abszurd abban ahogy több mint 20 ezer emberrel nézed egyszerre a történetet és a kommentek között minden kis történést ujjongva ünnepelnek az emberek.

Ha nem is tökéletes az így generált tartalom, a jövőben várható új médiatartalmakat így is elég jól előrejelzi, valószínűleg fél-egy éven belül már szinte PIXAR filmek látványvilágával rendelkező sorozatokat lehet így összerakni és ahogy látszik igény is lenne rá. Képzeljük csak el, hogy mi lenne ha a Netflixen gombonyásra generálhatnánk magunknak egy filmet a mi preferenciáinkat illetően, vagy akár valós időben interaktálhatnánk, befolyásolhatnánk a szereplők életét vagy a cselekményt. Én például szívesen megnéznék egy filmet az első hírben említett, most már háromlábú székről és hogy hogyan tervezi a bosszút. (https://www.twitch.tv/watchmeforever)

3. Blip2

Kicsit technikai hírek következnek, és ilyesmiket tervezek is a posztokba, hogy azoknak is hasznos legyen a hírlevél, akik amúgy napi szinten otthon vannak az AI témában.

Képzeljük el, hogy nem csak tisztán szöveggel kommunikálhatnánk a ChatGPT-vel, hanem küldhetnénk neki képeket és azok tartalmáról beszélhetnénk. Ez nem csak az instagram huszároknak segítene a tökéletes póz beállításában, de például tudományos cikkek ábráinak elemzését is el tudom képzelni mint lehetséges felhasználási lehetőség.

Tehát a ChatGPT és társai általában kizárólag szöveg-típusú információn lettek tanítva, de persze ez csak egy kis szelete a mindennapi világnak, és nem is alkalmas a teljes leírására. Hasznos lenne, ha már a modellek tanításánál használhatnánk képi információt, viszont ezt nem triviális hatékonyan kivitelezni. A címben említett BLIP2 egy olyan tanítási stratégia, ami ebben kíván segíteni. Majd ebbe egy másik posztban mennék bele részletesebben, de a lényeg, hogy modell betanítása után interaktívan kérdezhetünk egy adott képről, például itt ez az építészeti remekmű, hogy juthatunk ki belőle?

És tudta! De hasonlóan egy pizzát mutatva megkérdezhetjük tőle, hogy hogy kell megsütni, vagy tetszőleges virágot ábrázoló képre hogy milyen virág.

A következő linken bárki kipróbálhatja ingyenes, bármilyen fotón: https://huggingface.co/spaces/Salesforce/BLIP2

A hozzá tartozó cikk pedig elérhető itt: https://arxiv.org/abs/2301.12597

Ennyit gondoltam erre a hétre. A jövőben tervezek az ilyen jellegű posztokon kívül hosszabb, technikai leírásokat is írni, írjátok meg, hogy mik érdekelnének.

Share

Leave a comment

Subscribe now

1. Character.AI

It's not exactly a new thing, but now that ChatGPT has properly infiltrated public consciousness, it's worth mentioning the brilliant project called Character.AI, which has been under development for more than half a year. The model behind it similar to ChatGPT, so it's basically a chatbot, which we can outfit with its own personality with just a few sentences of description and a trial test conversation for fine tuning it.

Beyond simple applications, like having Einstein help solve your physics homework, or having Elon Musk give investment advice, you can quickly get to the more creative chatbots, like the "WhoWouldWin" pre-made personality who can tell you which of any two things would win, then you can argue with them if you disagree. I recommend comparisons like a stuck chewing gum on a shoe sole vs 67 liters of Starbucks coffee, it's pretty good. Or I had a deep conversation with a chair (!) one day, and after we argued about something, words led to blows and the poor thing had one of its legs smashed (he hit first), and it rewarded me with a soul-gripping monologue about human-chair power relationships for a few minutes before we finally made peace. Yes, I'll probably be the first person after the Boston Dynamics robot beaters to go before the AI court in the upcoming AI apocalypse (for those who don't know, here it is, but only for the brave:

).

A truly genius option is when you can unleash any number of chatbot personalities into a chatroom and they start talking to each other. I once put all the mentioned bots into a room and left them alone for 10 minutes and the most absurd, bizarre drama emerged from it.

So I can highly recommend trying out the site, it's constantly being developed but for now it's completely free. If you guys create an interesting AI personality, let me know in the comments below the post, I'm curious. (https://beta.character.ai/)

2. Nothing, Forever

Imagine a TV show or series where there are no pre-written dialogues or plot, but everything is generated in real-time by an artificial intelligence and the story changes organically at the same time. We can join the show at any time because the events are evolving 24/7. As the text-generating models shown earlier become more widespread and faster, it was only a matter of time for developers to experiment with this concept and the first attempt is the infinitely streaming show on Twitch, "Nothing, Forever."

Essentially, it's an infinite stream powered by an AI, similar to ChatGPT, based on the Seinfeld sitcom, where the text, the mandatory laughter track, the plot, and the camera movements are all generated by AI. Okay, the look is still frightening and reminiscent of the old PS1 graphics, but there's something absurdly exciting about watching the story with over 20,000 people at the same time and celebrating every little event with joy among the comments.

Even if the content generated this way is not perfect, it still predicts future media content quite well. In half a year or a year, it's likely that we'll be able to put together series with graphics as good as PIXAR films. And as it seems, there's demand for it. Imagine being able to generate a movie on Netflix based on our preferences, or even interact and influence the lives of the characters or the plot in real-time. I, for one, would love to watch a movie about the previously mentioned three-legged chair and its plan for revenge. (https://www.twitch.tv/watchmeforever)

3. Blip2

Up next are some technical news and I plan to include such things in my posts so that even those who are familiar with AI topics can find the newsletter useful.

Imagine that we could not only communicate with ChatGPT using text but also send it images and talk about their content. This would not only help Instagram enthusiasts get the perfect pose but I can also envision scientific article figures being analyzed as a potential use case.

So ChatGPT and its peers were only trained on text-type information, but of course this is only a small part of everyday life and not suitable for a complete description of reality. It would be useful if we could already use visual information when training the models, but this is not trivial to implement effectively. The BLIP2 mentioned in the title is a training strategy that aims to help with this. I will go into more detail in another post, but the essence is that after model training, we can interactively ask about a given image, such as this architectural masterpiece, how can we extract information from it?

And it knew it! Similarly, by showing him a pizza, we can ask him how to bake it, or any flower-representing picture what kind of flower it is.

Anyone can try it for free on any photo at the following link: https://huggingface.co/spaces/Salesforce/BLIP2

The accompanying article is available here: https://arxiv.org/abs/2301.12597

That's what I thought of this week. In the future, I plan to write longer, technical descriptions besides these types of posts. Let me know what would interest you.

Share

Leave a comment

Subscribe now

Subscribe now