Latest AI advances are aiming toward developing tabular foundation models (TFMs).
getty
In today’s column, I examine the rise of tabular foundation models (TFMs). These AI-based models are somewhat akin to large language models (LLMs), though TFMs are specifically designed to handle tabular data. Doing so is quite handy because conventional LLMs are not very good at dealing with tabular or spreadsheet data (regular LLMs tend to falter or fail when analyzing such data).
You see, conventional LLMs try to treat tabular data as though it can be entirely encoded into tokens, as though it is composed exclusively of text. TFMs are built to treat tabular data as it is intended to be treated, namely that numeric data is treated like numbers. A TFM identifies crucial patterns associated with data that resides in rows and columns. This is extremely useful for anyone scrutinizing spreadsheets or tabular data and hoping to lean into AI to help find valuable insights. I will walk you through the basics of TFMs. You also need to be aware of the upsides and downsides of when and how to use these powerful new AI models.
Let’s talk about it. This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
Setting Up Generative AI And LLMs
Before we leap into TFMs, some helpful background about LLMs will be useful to consider.
The typical way to set up generative AI and LLMs consists of first scanning lots of written data found across the Internet. All the well-known LLMs do this, including OpenAI’s ChatGPT and GPT-5, Anthropic’s Claude, Google’s Gemini, Microsoft’s Copilot, xAI’s Grok, and so on. Scanning allows the AI to pattern-match on human writing. The patterning is stored in a large-scale data structure loosely based on aspects of human wetware (vaguely like our brains), implemented as an artificial neural network (ANN). For more details on how this all works, see my in-depth discussion at the link here.
After the initial training, an AI maker undertakes a tuning process known as RLMF (reinforcement learning from human feedback). This consists of hiring human testers who give feedback to the AI. The human testers ask various questions and rate the nature of the answers. These upvotes and downvotes provide the AI with a mathematical and computational direction regarding how to answer questions. For example, if the testers are aiming to ensure that the AI is polite during interactions with users, they will upvote “please” and “thank you” (see my detailed explanation at the link here).
There are additional ways that an LLM is further tuned and shaped. An AI maker will usually provide a system-wide prompt to the AI that tells the AI how it is to act toward users. I have unpacked at great length various AI system-wide prompts; see the link here. The efforts by AI makers are intended to guide the AI in how it will behave once released and available to the public at large.
Difficulties With Tabular Data
People are often tempted to use LLMs to aid in exploring tabular data. A person might have a spreadsheet that contains business data and be interested in having an LLM find useful patterns in the data. They opt to upload the spreadsheet into the LLM and give a prompt that tells the AI to identify important relationships among the rows and columns of data.
There are many problems with doing this.
One notable aspect is that most LLMs try to examine the data by turning all of it into tokens that essentially represent text-oriented content (for details on how tokenization works, see my discussion at the link here). Suppose the spreadsheet contains the age of customers and the amount of merchandise they have purchased. Numbers such as an age of 36 and a purchase total of $8,500 would be converted into tokens as though those aren’t numbers. The number 36 is composed of the character “3” followed by the character “6”. You and I know that this is supposed to represent a numeric value and not simply two adjacent text characters.
The AI is going to have a challenging time trying to figure out any mathematical relationships between the customer age and purchases since the data isn’t being codified as numeric per se. This will also greatly impact important numerical elements such as precision and accuracy. The odds are that the LLM is going to make various mistakes by not treating the data as numeric.
Tabular Foundation Models (TFMs)
The idea then is to start with a clean slate and devise an AI that will expediently handle tabular data. We will reuse many of the same practices and techniques that go into making an LLM. A key difference is that we want the TFM to be ready-made for numeric data, and that the data will be housed in rows and columns, and that the data will likely contain a mixture of both numeric and text-based information.
On top of that, we want the TFM to deal properly with the typical types of data issues and errors that arise in raw data. This includes missing values, values that are oddly out of range, values that are not numeric that are in an otherwise numeric field, and so on. The TFM should be looking for data anomalies that might exist anywhere in the tabular data.
We also want the TFM to be figuring out relationships among rows and columns. Are there correlations that stand out? Does the numeric distribution of the value in a row or column have any noticeable significance? Are there any seasonal effects in the data? The gist is that the TFM must be a statistically savvy machine that looks for substantive mathematical relationships, and that doesn’t get easily fooled by misleading or false relationships.
Training Of TFM
I had earlier mentioned that an LLM is principally data-trained by scanning the Internet widely for human-written content. For a TFM, we might imagine that the same could be done by scanning spreadsheets and other tabular data that is largely found across the Internet. The assumption is that what works for the goose ought to work for the gander.
Unfortunately, that’s not as easy as it sounds. Trying to scan for tabular data is more challenging than finding textual content. The tabular data might be tossed together with no rhyme or reason. A lot of it is pure garbage. There isn’t any usefulness to the data. In contrast, when training an LLM, most text-based content is usually of some sensible nature and can be taken at face value.
Ideally, we would want to find tabular data that perhaps has some associated text that explains what the data is all about. If we could find text that provides an interpretation about the data, this would be especially advantageous. The TFM could compare the tabular data to the text and use that to gain an added semblance of what people look for in tabular data.
The bottom line is that rather than sourcing data from the Internet, TFMs are often trained by creating data that is specifically crafted for TFM training purposes. An AI developer first establishes many sets of tabular data and gets that data ready for doing the TFM training. An astute developer seeks to shape data that is realistic, containing blanks, oddball values, etc. This made-up or synthetic data ought to closely resemble real-world data. It will be of limited value to simply pump pristine data into a TFM during training. The TFM will not be prepared to handle real-world circumstances.
Latest Examples Of TFM
The currently popular TFMs include TabFM (Google Research), TabPFN (Prior Labs), TabICL (SODA), KumoRFM (Nvidia/Kumo), and others that are also in the works. At this stage, it is usually undertaken as part of an advanced AI research effort and/or pursued by plucky startups.
All told, this is an evolving area of research and practice. I highlight that point to emphasize that TFMs are still in their infancy. If you opt to use a TFM, do so with a mindset that these new types of AI models are evolving and that you might have to be highly attentive. The chances are you’ll need to do some heavy lifting. These aren’t pushbuttons just yet.
There is also an aim to create TFM’s that are either across-the-board or devised for specific domains. Here’s what I mean. The TFM could be shaped to support any domain of interest, such as financial tabular data, health-related tabular data, inventory tabular data, customer service tabular data, etc. A broad-based TFM will try to be a jack of all trades.
Sometimes, a broad-based TFM cannot do as good a job in a specific domain as would a TFM that was tailored for the domain of interest. For example, an AI developer could concentrate on making a TFM that handles primarily medical tabular data. That’s what this particular TFM does best. Anyone using it should realize that angling the TFM to other domains might be possible, but probably it would be wiser to find a TFM targeted at that other domain or one that is considered broad-based.
Latest Research On TFMs
Let’s take a brief look at one TFM, namely TabFM. In a recently released research paper entitled “Introducing TabFM: A Zero-Shot Foundation Model For Tabular Data” by Weihao Kong, Abhimanyu Das, Google Research, June 30, 2026, these salient points were made (excerpts):
- “Today, we introduce TabFM, a foundation model designed specifically for tabular data classification and regression.”
- “Tabular data constitutes the backbone of enterprise data infrastructure and powers a significant fraction of critical predictive machine learning applications.”
- “Standard language models process one-dimensional, ordered sequences, but tables are fundamentally two-dimensional and inherently orderless: swapping two rows or two columns does not change the underlying meaning of the data.”
- “TabFM is trained entirely on hundreds of millions of synthetic datasets.”
- “As a result, the model generalizes well to unseen real-world tables, as we demonstrate in our benchmarks.”
This TFM is one of the now classics in this arena and continues to be updated and enhanced. If the topic of TFM interests you, I’d urge you to consider reading the above-cited paper. The researchers opted to use a multilayer attention module and abide by a clever method of alternating attention across rows and columns. The approach includes row compression and extensively uses in-context learning (ICL).
The Future Is Hybrid
My prediction is that we are going to see not only advancement in TFMs, but we will also end up connecting and combining LLMs and TFMs. You will log into your everyday LLM, and when asking to have the AI aid in tabular data analysis, the LLM will connect with a TFM and use that behind the scenes for you. The LLM will be your natural language interface and the TFM will be the tabular data workhorse.
I have been predicting that the future of AI is going to be hybrid foundational models, including neuro-symbolic foundation models, LLM-TFM hybrids, and so on. See my in-depth predictions at the link here.
A final thought for now. The famous mathematician and computer pioneer Ada Lovelace made this notable remark: “One essential object is to choose that arrangement which shall tend to reduce to a minimum the time necessary for completing the calculation.” I would contend that using TFMs to aid in analyzing your tabular data is a befitting example of choosing the right arrangement for the delicate matter at hand.
Leave a comment