Databricks, a company that helps large enterprises build custom AI models, has developed a machine learning technique that can improve the performance of AI models without cleaning tagged data.
Databricks chief AI scientist Jonathan Frankle talked with clients over the past year about the key challenges they face in making AI reliable.
Frank said the problem was dirty data.
“Everyone has some data and knows what they want to do,” Frank said. However, the lack of clean data makes it challenging to fine-tune the models that perform a specific task. “No one shows up beautiful, clean fine-tuning data, you can stick it to the prompt or [application programming interface]”For the model.
Databricks’ model can allow companies to eventually deploy their own agents to perform tasks without hindering data quality.
The technology rarely looks at some of the key tips engineers are using now to improve the capabilities of advanced AI models, especially when it is difficult to get good data. This approach takes advantage of the idea of helping to generate advanced inference models by combining enhanced learning, an approach to AI models that improve, “synthesize” or AI-generated training data.
The latest models from OpenAI, Google and DeepSeek all rely heavily on reinforcement learning as well as synthetic training data. Wired revealed that NVIDIA plans to acquire Gretel, a company specializing in synthetic data. “We’re all browsing the space,” Frank said.
The Databricks method takes advantage of the fact that given enough attempts, even weak models can score well on a given task or benchmark. The researchers call this method to improve model performance “optimal N.” Databricks trained a model to predict which best results human testers prefer based on examples. Datamap reward model or DBRM can then be used to improve the performance of other models without further labeling of data.
Then use DBRM to select the best output from the given model. This creates synthetic training data to further fine-tune the model, resulting in better output for the first time. Databricks says its new method tests time adaptive optimization or TAO. “The approach we’re talking about uses some relatively light reinforcement learning learning, from baking the benefits of the best-N into the model itself,” Frank said.
He added that the research done by Databricks shows that the TAO method is extended to a larger, more powerful model. Reinforcement learning and synthetic data have been widely used, but combining them to improve language models is a relatively new and technically challenging technique.
Databricks is extremely open to how it develops AI because it wants to show customers that it has the skills it needs to create a powerful custom model for them. The company previously revealed to Wired how to develop DBX, a cutting-edge open source big-word model (LLM) from scratch.
