.svg){kind=small}
Training and Sampling
In this guide, we'll walk you through using the Bios Python library to perform the core operations needed for training and sampling from UltraSafe expert models.
📚 This quickstart demonstrates basic API usage. For production best practices and advanced techniques, check out our Training Guides and Bios Cookbook examples.
Creating the Training Client
The main object you'll work with is the TrainingClient, which represents a fine-tuned model that you can train and sample from.
Set Your API Key
First, set your Bios API key as an environment variable:
1# In your terminal or .bashrc
2export BIOS_API_KEY=<your-api-key>Initialize the Service Client
Create a ServiceClient to discover available UltraSafe models:
1import bios
2
3service_client = bios.ServiceClient()
4print("Available UltraSafe models:")
5for item in service_client.get_server_capabilities().supported_models:
6 print("- " + item.model_name)You'll see a list of UltraSafe expert models:
1- ultrasafe/usf-mini
2- ultrasafe/usf-healthcare
3- ultrasafe/usf-finance
4- ultrasafe/usf-code
5- ultrasafe/usf-conversationNote: Bios exclusively supports UltraSafe's closed-source expert models. These models are purpose-built for specific domains with enterprise-grade security and compliance.
Create the Training Client
Now create a training client for the model you want to fine-tune. We'll use the Finance expert model for this example:
1base_model = "ultrasafe/usf-finance"
2training_client = service_client.create_lora_training_client(
3 base_model=base_model
4)Preparing the Training Data
For this example, we'll train a specialized model for financial terminology translation. This is a simple demonstration of the API—for production use cases, see our advanced guides.
Create Training Examples
Let's create training examples that teach the model financial term definitions:
1# Create financial terminology training examples
2examples = [
3 {
4 "term": "bull market",
5 "definition": "A period of rising stock prices and investor optimism"
6 },
7 {
8 "term": "bear market",
9 "definition": "A period of declining stock prices and investor pessimism"
10 },
11 {
12 "term": "liquidity",
13 "definition": "The ease with which an asset can be converted to cash"
14 },
15 {
16 "term": "diversification",
17 "definition": "Spreading investments across different assets to reduce risk"
18 },
19 {
20 "term": "dividend yield",
21 "definition": "Annual dividend payment divided by stock price"
22 },
23 {
24 "term": "market capitalization",
25 "definition": "Total value of a company's outstanding shares"
26 },
27 {
28 "term": "volatility",
29 "definition": "The degree of variation in asset prices over time"
30 },
31]Process Examples for Training
Convert the examples into the format expected by the Bios training client:
1from bios import types
2
3# Get the tokenizer from the training client
4tokenizer = training_client.get_tokenizer()
5
6def process_example(example: dict, tokenizer) -> types.Datum:
7 # Format the input with Term/Definition template
8 prompt = f"Financial Term: {example['term']}\nDefinition:"
9
10 prompt_tokens = tokenizer.encode(prompt, add_special_tokens=True)
11 prompt_weights = [0] * len(prompt_tokens)
12
13 # Add space before output and finish with double newline
14 completion_tokens = tokenizer.encode(
15 f" {example['definition']}\n\n",
16 add_special_tokens=False
17 )
18 completion_weights = [1] * len(completion_tokens)
19
20 tokens = prompt_tokens + completion_tokens
21 weights = prompt_weights + completion_weights
22
23 input_tokens = tokens[:-1]
24 target_tokens = tokens[1:] # Predict next token (shifted)
25 weights = weights[1:]
26
27 # Create a Datum - a single training example
28 return types.Datum(
29 model_input=types.ModelInput.from_ints(tokens=input_tokens),
30 loss_fn_inputs=dict(weights=weights, target_tokens=target_tokens)
31 )
32
33processed_examples = [process_example(ex, tokenizer) for ex in examples]Visualize Training Data
Inspect the first example to verify correct formatting:
1# Visualize the first example
2datum0 = processed_examples[0]
3print(f"{'Input':<20} {'Target':<20} {'Weight':<10}")
4print("-" * 50)
5for inp, tgt, wgt in zip(
6 datum0.model_input.to_ints(),
7 datum0.loss_fn_inputs['target_tokens'].tolist(),
8 datum0.loss_fn_inputs['weights'].tolist()
9):
10 print(f"{repr(tokenizer.decode([inp])):<20} {repr(tokenizer.decode([tgt])):<20} {wgt:<10}")The output shows token-level details with weights (0 for prompt, 1 for completion):
Input Target Weight -------------------------------------------------- 'Financial' ' Term' 0.0 ' Term' ':' 0.0 ':' ' bull' 0.0 ' bull' ' market' 0.0 ' market' '\n' 0.0 '\n' 'Definition' 0.0 'Definition' ':' 0.0 ':' ' A' 1.0 ' A' ' period' 1.0 ' period' ' of' 1.0 ' of' ' rising' 1.0 ' rising' ' stock' 1.0 ' stock' ' prices' 1.0 ...
Performing Training Updates
Now we can use this data to perform training updates with the simplified train() API. We'll run 6 training iterations on our batch, with all computation executing on UltraSafe's GPU cloud.
1import numpy as np
2
3for iteration in range(6):
4 # Single train() call handles everything on UltraSafe's cloud
5 result = training_client.train(
6 processed_examples,
7 loss_fn="cross_entropy",
8 learning_rate=1e-4
9 )
10
11 # Extract loss from result
12 loss = result.loss
13 print(f"Iteration {iteration + 1}: Loss per token = {loss:.4f}")Cloud-Based Execution
The train() function abstracts all training mechanics including forward pass, backpropagation, and optimization. All computation executes exclusively on UltraSafe's proprietary GPU cloud infrastructure, eliminating the need for local resources.
Expected output showing decreasing loss:
Iteration 1: Loss per token = 3.2847 Iteration 2: Loss per token = 2.9123 Iteration 3: Loss per token = 2.6891 Iteration 4: Loss per token = 2.4256 Iteration 5: Loss per token = 2.1834 Iteration 6: Loss per token = 1.9672
Sampling from the Model
Now we can test our fine-tuned model by generating definitions for financial terms.
Create Sampling Client
First, save the trained weights and create a sampling client:
1# Save weights and get sampling client
2sampling_client = training_client.save_weights_and_get_sampling_client(
3 name='finance-terminology-model'
4)Generate Predictions
Test the model by asking it to define a financial term:
1# Prepare prompt
2prompt = types.ModelInput.from_ints(
3 tokenizer.encode("Financial Term: portfolio\nDefinition:")
4)
5
6# Configure sampling parameters
7params = types.SamplingParams(
8 max_tokens=50,
9 temperature=0.0, # Greedy sampling for deterministic output
10 stop=["\n"]
11)
12
13# Generate samples
14future = sampling_client.sample(
15 prompt=prompt,
16 sampling_params=params,
17 num_samples=4
18)
19result = future.result()
20
21# Display results
22print("Model Responses:")
23for i, seq in enumerate(result.sequences):
24 decoded = tokenizer.decode(seq.tokens)
25 print(f"{i + 1}: {repr(decoded)}")Example output from the fine-tuned model:
Model Responses: 1: ' A collection of financial investments held by an investor\n' 2: ' A group of investments managed together to achieve financial goals\n' 3: ' An investor's collection of stocks, bonds, and other securities\n' 4: ' A diversified set of assets owned by an individual or institution\n'
Note on Sampling
Even with temperature=0.0, sampling can be slightly nondeterministic due to distributed batching. For production use, consider using beam search or other deterministic decoding strategies.
Complete Example Script
Here's the complete training and sampling workflow in one script:
1#!/usr/bin/env python3
2"""
3Complete Bios Quick Start Example
4Train and sample from UltraSafe Finance model
5"""
6
7import bios
8from bios import types
9import numpy as np
10
11# Initialize service client
12service_client = bios.ServiceClient()
13
14# Create training client
15training_client = service_client.create_lora_training_client(
16 base_model="ultrasafe/usf-finance"
17)
18
19# Get tokenizer
20tokenizer = training_client.get_tokenizer()
21
22# Prepare training data
23examples = [
24 {"term": "bull market", "definition": "A period of rising stock prices"},
25 {"term": "bear market", "definition": "A period of declining stock prices"},
26 # ... more examples
27]
28
29def process_example(example, tokenizer):
30 prompt = f"Financial Term: {example['term']}\nDefinition:"
31 # ... processing logic (see above)
32 return datum
33
34processed_examples = [process_example(ex, tokenizer) for ex in examples]
35
36# Train for 6 iterations
37for i in range(6):
38 fwdbwd_future = training_client.forward_backward(processed_examples, "cross_entropy")
39 optim_future = training_client.optim_step(types.AdamParams(learning_rate=1e-4))
40 fwdbwd_result = fwdbwd_future.result()
41 # ... compute loss
42 print(f"Iteration {i + 1}: Loss = {loss:.4f}")
43
44# Save and sample
45sampling_client = training_client.save_weights_and_get_sampling_client(
46 name='finance-model'
47)
48prompt = types.ModelInput.from_ints(
49 tokenizer.encode("Financial Term: portfolio\nDefinition:")
50)
51result = sampling_client.sample(
52 prompt=prompt,
53 sampling_params=types.SamplingParams(max_tokens=50, temperature=0.0, stop=["\n"]),
54 num_samples=4
55).result()
56
57for i, seq in enumerate(result.sequences):
58 print(f"{i + 1}: {tokenizer.decode(seq.tokens)}")Next Steps
Now that you understand the basics, explore advanced training techniques and best practices: