Re-using fine-tuned models

Reusing previously fine-tuned TimeGPT models can help reduce computation time and costs while maintaining or improving forecast accuracy. This guide walks you through the steps to save, fine-tune, list, and delete your TimeGPT models effectively.

How to Re-use Fine-tuned Models

Step 1: Import Packages

First, we import the required packages and initialize the Nixtla client

import pandas as pd
from nixtla import NixtlaClient
from utilsforecast.losses import rmse
from utilsforecast.evaluation import evaluate

nixtla_client = NixtlaClient(
    # defaults to os.environ["NIXTLA_API_KEY"]
    api_key='my_api_key_provided_by_nixtla'
)

Step 2: Load Data

Load the forecasting dataset and prepare the train/validation split.

df = pd.read_parquet('https://datasets-nixtla.s3.amazonaws.com/m4-hourly.parquet')

h = 48

valid = df.groupby('unique_id', observed=True).tail(h)
train = df.drop(valid.index)

train.head()

	unique_id	ds	y
0	H1	1	605.0
1	H1	2	586.0
2	H1	3	586.0
3	H1	4	559.0
4	H1	5	511.0

Step 3: Zero-shot forecast

We can try forecasting without any finetuning to see how well TimeGPT does.

fcst_kwargs = {
    'df': train,
    'freq': 1,
    'model': 'timegpt-1-long-horizon'
}

fcst = nixtla_client.forecast(h=h, **fcst_kwargs)

zero_shot_eval = evaluate(fcst.merge(valid), metrics=[rmse], agg_fn='mean')
zero_shot_eval

metric	TimeGPT
rmse	1504.474342

Step 4: Fine-tune the model

We can now fine-tune TimeGPT a little and save our model for later use. We can define the ID that we want that model to have by providing it through output_model_id. This ID is also returned as the output of the finetune method.

first_model_id = 'my-first-finetuned-model'

nixtla_client.finetune(output_model_id=first_model_id, **fcst_kwargs)

'my-first-finetuned-model'

We can now forecast using this fine-tuned model by providing its ID through the finetuned_model_id argument.

first_finetune_fcst = nixtla_client.forecast(
    h=h,
    finetuned_model_id=first_model_id,
    **fcst_kwargs
)

first_finetune_eval = evaluate(
    first_finetune_fcst.merge(valid),
    metrics=[rmse],
    agg_fn='mean'
)

zero_shot_eval.merge(
    first_finetune_eval,
    on=['metric'],
    suffixes=('_zero_shot', '_first_finetune')
)

metric	TimeGPT_zero_shot	TimeGPT_first_finetune
rmse	1504.474342	1472.024619

We can see the error was reduced.

Step 5: Further fine-tune the model

We can now take this model and fine-tune it a bit further by using the NixtlaClient.finetune method but providing our already fine-tuned model as finetuned_model_id, which will take that model and fine-tune it a bit more. We can also change the fine-tuning settings, like using finetune_depth=3, for example. As before, the new finetuned model ID is returned by the finetune method.

second_model_id = nixtla_client.finetune(
    finetuned_model_id=first_model_id,
    finetune_depth=3,
    **fcst_kwargs
)

second_model_id

'468b13fb-4b26-447a-bd87-87a64b50d913'

Since we didn’t provide output_model_id this time, it got assigned an UUID. We can now use this model to forecast.

second_finetune_fcst = nixtla_client.forecast(
    h=h,
    finetuned_model_id=second_model_id,
    **fcst_kwargs
)

second_finetune_eval = evaluate(
    second_finetune_fcst.merge(valid),
    metrics=[rmse],
    agg_fn='mean'
)

first_finetune_eval.merge(
    second_finetune_eval,
    on=['metric'],
    suffixes=('_first_finetune', '_second_finetune')
)

metric	TimeGPT_first_finetune	TimeGPT_second_finetune
rmse	1472.024619	1435.365211

We can see the error was reduced a bit more.

Step 6: List fine-tuned models

We can list our fine-tuned models with the NixtlaClient.finetuned_models method.

finetuned_models = nixtla_client.finetuned_models()
finetuned_models

[FinetunedModel(id='468b13fb-4b26-447a-bd87-87a64b50d913', created_at=datetime.datetime(2024, 12, 30, 17, 57, 31, 241455, tzinfo=TzInfo(UTC)), created_by='user', base_model_id='my-first-finetuned-model', steps=10, depth=3, loss='default', model='timegpt-1-long-horizon', freq='MS'),
 FinetunedModel(id='my-first-finetuned-model', created_at=datetime.datetime(2024, 12, 30, 17, 57, 16, 978907, tzinfo=TzInfo(UTC)), created_by='user', base_model_id='None', steps=10, depth=1, loss='default', model='timegpt-1-long-horizon', freq='MS')]

While that representation may be useful for programmatic use, in this exploratory setting it’s nicer to see them as a dataframe, which we can get by providing as_df=True.

nixtla_client.finetuned_models(as_df=True)

id	created_at	created_by	base_model_id	steps	depth	loss	model	freq
468b13fb-4b26-447a-bd87-87a64b50d913	2024-12-30 17:57:31.241455+00:00	user	my-first-finetuned-model	10	3	default	timegpt-1-long-horizon	MS
my-first-finetuned-model	2024-12-30 17:57:16.978907+00:00	user	None	10	1	default	timegpt-1-long-horizon	MS

We can see that the base_model_id of our second model is our first model, along with other metadata.

Step 7: Delete fine-tuned models

In order to keep things organized, and since there’s a limit of 50 fine-tuned models, you can delete models that weren’t so promising to make room for more experiments. For example, we can delete our first finetuned model. Note that even though it was used as the base for our second model, they’re saved independently so removing it won’t affect our second model, except for the dangling metadata.

nixtla_client.delete_finetuned_model(first_model_id)

nixtla_client.finetuned_models(as_df=True)

id	created_at	created_by	base_model_id	steps	depth	loss	model	freq
468b13fb-4b26-447a-bd87-87a64b50d913	2024-12-30 17:57:31.241455+00:00	user	my-first-finetuned-model	10	3	default	timegpt-1-long-horizon	MS

WARNING: Deleting a fine-tuned model is irreversible. Make sure to back up any necessary information before removal.

Conclusion

Congratulations! You have successfully learned how to save, refine, and manage your fine-tuned TimeGPT models. This workflow helps optimize your forecasting pipelines by leveraging previously generated insights.

INTRODUCTION

SETUP

DATA REQUIREMENTS

FORECASTING

ANOMALY DETECTION

USE CASES

REFERENCE

About

Re-using fine-tuned models