Open Models Are Becoming Research Infrastructure

Llama 3.1 is being discussed mainly as a model release: 8B, 70B and a new 405B parameter model, with 128K context and competitive results against leading closed systems.

I think the more interesting change is what Meta is allowing people to do with the model.

The 405B weights are available under the Llama 3.1 Community License, and Meta has changed the license so outputs from Llama models can be used to improve other models. Meta explicitly points to synthetic data generation and model distillation as target workflows.

That changes the role of an open model. It is no longer only a cheaper or more controllable alternative to a hosted API. A capable open-weight model can become infrastructure for building the next model, dataset, evaluator or application stack.

The model is not just the thing you deploy

Most comparisons between open and closed models focus on inference.

They ask whether an open model answers the same questions, how much GPU memory it needs, whether it is cheaper to host, and how far behind GPT-4 or Claude it sits on benchmarks.

Those questions matter for deployment, but they miss one property that becomes more valuable as open models improve: access changes what kinds of experiments are possible.

With a closed API, I can change prompts, retrieval, tool definitions and application logic. I cannot modify the weights, quantise the model for my own hardware, train directly on top of it under my own process, or assume the provider allows its outputs to become training material for another model.

An open-weight model changes that boundary.

Meta says Llama 3.1 developers can download the weights, customise the models, train on new datasets and perform further fine-tuning. The model card explicitly permits using model outputs to improve other models through synthetic data generation and distillation.

That last part looks like a licensing detail. I think it is one of the most consequential parts of the release.

A frontier-scale model can become a teacher

Training a 405B model from scratch is far outside the budget of most teams. Meta says Llama 3.1 405B was trained on more than 15 trillion tokens using over 16,000 H100 GPUs.

Using a trained 405B model is a different problem.

A smaller lab or company can use a large model as a teacher: generate answers, reasoning examples, classifications or task-specific data, then train a smaller model on those outputs. This is the basic idea behind model distillation.

Meta is using a version of that workflow itself. Its Llama 3.1 release says the 405B model was used to improve the post-training quality of the smaller models. Meta says synthetic data forms the majority of its supervised fine-tuning examples during post-training, with filtering and repeated rounds used to improve data quality.

The expensive step is producing the general model. Once its weights and outputs are accessible, other teams can reuse some of that capability without reproducing the original training run.

For many product teams, the useful artifact may therefore not be the 405B model running in production. It may be a dataset generated with it, a smaller model trained from that dataset, or an evaluation system built around it.

Synthetic data changes what "access" means

Synthetic data is often described as a way to produce more examples when human data is scarce. A strong teacher can do something more targeted: generate data around a specific production problem.

For a development workflow, that could mean examples of code transformations, structured tool calls or repository tasks. For a multilingual game pipeline, it could mean candidate localisation pairs that are filtered and reviewed before training a smaller model. In a 3D pipeline, the same pattern could produce instruction-to-operation examples around naming, scene validation, asset metadata or Blender automation.

I would not treat model-generated data as ground truth. The teacher's errors can become training data too.

This makes filtering, validation and dataset design part of the model pipeline. Meta's own training description reflects that: synthetic examples go through filtering, rejection sampling and repeated post-training rounds.

The model provides scale. The pipeline decides what survives.

That moves some AI engineering away from prompt design and toward data engineering.

Open does not mean unconstrained

There is a terminology problem here.

Meta calls Llama 3.1 open source. I find "open-weight" more precise for this discussion. The weights can be downloaded and modified, but the release uses Meta's custom Llama 3.1 Community License. The license contains conditions, including separate terms for products with more than 700 million monthly active users.

The release still opens a much larger experimental surface than a model available only through an API.

Researchers and developers can run the same model artifact on infrastructure they control, fine-tune it, alter inference systems around it and build derivatives. Hardware experiments become possible. Quantisation work becomes possible. Fine-tuning recipes can be compared against the same base.

Meta says more than 25 partners supported Llama 3.1 at launch, spanning clouds, inference providers and hardware vendors. It says it worked with projects including vLLM, TensorRT and PyTorch for day-one support.

A model starts to look like infrastructure when tooling can form around it.

The research value may exceed the deployment value

A 405B dense model is not a practical local model for most developers.

Meta quantised its production version from BF16 to FP8 to lower inference requirements, yet this remains a very large system. For many teams, the 8B or 70B variants will be easier to deploy.

This is why I do not think the main question is whether everyone will run 405B.

The better question is what becomes possible when a frontier-scale teacher is available to the ecosystem.

A team might use 405B offline to generate domain data, adapt an 8B model with those examples, run the smaller model locally, then evaluate it against deterministic tests and a stronger model. Another team might study quantisation. Another might build a new fine-tuning method or generate adversarial evaluation cases.

The model becomes an input to research rather than only the final product of research.

What this means for builders

For founders and developers, I would separate model selection into two decisions.

The first is the production model: what should answer requests at runtime given latency, privacy, cost and hardware constraints?

The second is the development model: what should help create data, evaluate behaviour, test workflows and train the production system?

They do not need to be the same model.

That separation is familiar from rendering and game production. The highest-quality representation is not always the representation shipped to the user. We bake lighting, generate LODs, compress textures and convert source assets into runtime formats. The heavy production artifact exists partly to generate efficient downstream artifacts.

Large open-weight models may take a similar role in AI pipelines.

A 405B model can be the high-quality source asset. Smaller models become runtime derivatives shaped around the actual product.

Where I think this goes

If open models remain close enough to the frontier, the surrounding ecosystem may become more important than small differences on leaderboards.

Meta is already proposing Llama Stack as a common set of interfaces for inference, fine-tuning, synthetic data generation and agentic applications. Whether that specific interface becomes a standard is uncertain. The direction makes sense: once models become building blocks, teams need repeatable tooling around training, evaluation and serving.

I expect more companies to use large models as internal teachers rather than deploy them directly for every request. Synthetic datasets, domain-specific smaller models and model-based evaluation can become normal pieces of a production pipeline.

There is a limit to this prediction. A teacher can reproduce its own blind spots, and synthetic data can collapse diversity if teams generate and train without strong filtering. Access to weights does not solve data quality, evaluation or compute constraints.

But the category has changed.

An open model used to mean a model you could run yourself.

Llama 3.1 suggests a broader role: a model you can build other systems and other models from.

That is much closer to research infrastructure than a chatbot alternative.

Sources

1. Meta AI, Introducing Llama 3.1: Our most capable models to date, 23 July 2024.
2. Meta AI, The Llama 3 Herd of Models, 23 July 2024.
3. Meta, Llama 3.1 Community License Agreement, 23 July 2024.
4. Meta, Llama 3.1 Model Card, 23 July 2024.
5. Mark Zuckerberg / Meta, Open Source AI Is the Path Forward, 23 July 2024.