Keep a fine-tuned model working after its provider retires the base model.
OpenAI shuts down fine-tuned gpt-4o, gpt-4o-mini, gpt-4.1 and gpt-4.1-mini models on
14 January 2027, and stops new fine-tuning jobs on 6 January 2027. The named replacement,
gpt-6-luna, cannot be fine-tuned. Once a model is gone, so is the only exact record of how it behaved.
tunekeep helps you get out in time:
| Step | Command |
|---|---|
| 1. See every fine-tune you own and its days left | tunekeep inventory |
| 2. Record how each model answers, as a golden eval set | tunekeep record |
| 3. Check whether gpt-6-luna (or any model) already matches it | tunekeep compare |
| 4. Retrain an open model with LoRA from your own data | tunekeep retrain |
| 5. One HTML report: quality, speed and tokens side by side | tunekeep report |
Do the recording first. Everything else can happen later, but recording cannot happen after the shutdown.
uv tool install tunekeep # or: pipx install tunekeepPython 3.10+. No dependencies: tunekeep uses only the Python standard library. (The retraining kit it writes has its own pinned requirements, installed only where you run it.)
export OPENAI_API_KEY=sk-... # a key for the org/project that owns the fine-tunes
tunekeep inventoryFine-tuned models: 1 (plus 1 checkpoint)
New fine-tuning jobs stop on 2027-01-06 (60 days).
MODEL SHUTS DOWN DAYS LEFT SOURCE
ft:gpt-4o-mini-2024-07-18:acme::AbC123 2027-01-14 68 api
Files: 1 saved, 1 training/validation file the API would not return.
Keep your own copy of these; tunekeep retrains only from your original data:
- file-train-0 train.jsonl (job ftjob-000)
Next: record how each model behaves while it still answers, e.g.
tunekeep record --model ft:gpt-4o-mini-2024-07-18:acme::AbC123 --prompts prompts.jsonl
tunekeep-out/inventory.json holds the full detail: jobs, hyperparameters, checkpoints and their
metrics, and every file. Result files (training metrics) are downloaded. Files you uploaded for
training are not: OpenAI does not return them through the API, so keep your own copy.
tunekeep record --model ft:gpt-4o-mini-2024-07-18:acme::AbC123 --prompts prompts.jsonlprompts.jsonlcan be your training file as-is. The final assistant message on each line is removed before sending and kept as the reference answer. DPO lines (input.messages) work too.- Real production prompts make the best golden set. Use a few hundred, covering your real traffic.
- Answers are recorded at
temperature 0by default. Use--seedfor more repeatable runs. - Safe to stop and re-run: prompts already recorded are skipped, failed ones are retried.
--dry-runcounts prompts and estimated input tokens without sending anything.
tunekeep compare --golden tunekeep-out/golden-ft_gpt-4o-mini-2024-07-18_acme_AbC123.jsonl \
--candidate gpt-6-luna --examples train.jsonl --shots 5Each recorded prompt goes to the candidate, optionally with a few examples from your own training file, and the answer is scored against the recorded one:
- Tool calls: matched by function name, arguments compared field by field.
- JSON answers: the share of leaf fields with the same value. Invalid JSON scores 0.
- Text: 100% when equal after normalising case and spacing, otherwise word overlap (F1). Overlap is a rough signal for free text, so the report shows the biggest differences side by side.
GPT-6 Luna reasons by default, and in Chat Completions it accepts temperature and function tools
only with reasoning_effort: "none". tunekeep sends none by default (--reasoning-effort auto),
which is also closest to how a fine-tuned gpt-4o-mini answered. Try --reasoning-effort low to see
whether reasoning helps. tunekeep then drops temperature, and refuses before sending anything if your
prompts use tools.
Any OpenAI-compatible endpoint works as a candidate: --candidate-base-url http://127.0.0.1:8080/v1
for llama-server, vLLM or Ollama (no key needed on localhost, --candidate-key-env for others).
tunekeep retrain --data train.jsonl # default base: Qwen/Qwen3-1.7B, pinned revision
cd tunekeep-out/retrain && modal run modal_app.pyretrain converts your OpenAI training file (chat format, tools, legacy functions, per-message
weight) and writes a small kit: your data, a config, a training script and a Modal
app. It uploads nothing itself. The kit:
- trains a LoRA adapter with loss only on assistant messages (
"weight": 0turns are skipped, as on OpenAI) - reuses your original job's epoch count when
tunekeep-out/inventory.jsonhas it - pins every package by hash, the base model by revision, and llama.cpp by commit
- stops after a time limit (default 2 hours), which caps the cost of a run
- exports merged weights and a q8_0 GGUF for llama-server or Ollama
Use --check to only validate and summarise the data. See the kit's own README for running it
elsewhere and serving the result. Not yet supported: DPO, images and audio in training data.
A public proof run (examples/banking77): bank customer messages, 77 intents,
JSON answers. Qwen3-1.7B went from 50.3% to 90.6% accuracy on 770 unseen messages after
tunekeep retrain (3,075 examples, 8.5 minutes on one L40S), served locally from the exported
GGUF and scored with tunekeep compare.
tunekeep report # reads tunekeep-out/, writes tunekeep-out/report.htmlOne self-contained HTML file with deadlines, every candidate's scores, latency and token use, and the lowest-scoring answers side by side. It loads nothing from the network and runs no scripts, and every value from your data is escaped.
It never trains on recorded outputs. OpenAI's Services Agreement (section 3.3(e)) restricts using Output to develop competing models. Recorded answers are used only to evaluate a replacement. Retraining uses only your original training data. If that data was itself generated by an OpenAI model, it counts as Output too. Check with your own counsel. This is not legal advice.
- tunekeep talks only to the API base URLs you give it. No telemetry, no other network calls.
- Outputs can contain production prompts, so the output folder is created
0700and files0600. - API keys are read from the environment and are never written to disk or printed. A candidate
endpoint never receives your OpenAI key unless you name that variable with
--candidate-key-env.
--base-url/OPENAI_BASE_URLfor OpenAI-compatible gateways.--azure --base-url https://<resource>.openai.azure.com/openai/v1withAZURE_OPENAI_API_KEY(experimental). Azure retires fine-tuned gpt-4o and gpt-4o-mini deployments on 1 October 2027, and the gpt-4.1 family on 14 October 2027.
python -m unittest discover -s tests -vThe tests run against a fake OpenAI server built from the public API reference: pagination, the
shutdown_date field, blocked training-file downloads, rate limits, tool calls and GPT-6 Luna's
parameter rules. Both the fake server's responses and every request tunekeep sends are checked
against OpenAI's official API spec (openai-openapi,
pinned by commit and SHA-256, see tools/openai_spec.py), so the tests notice when either drifts
from the real API. The retraining kit is tested end to end on CPU with Qwen3-0.6B.
tools/lock-kit.sh regenerates the kit's hash-locked requirements.
Reports from real accounts are very welcome. Please open an issue with anything that behaves differently.
- OpenAI deprecations: https://developers.openai.com/api/docs/deprecations
- GPT-6 parameters: https://developers.openai.com/api/docs/guides/latest-model
- Azure retirement schedule: https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-retirement-schedule
Apache-2.0. Built by Rush Labs, an applied AI lab.
