Specksim example
The Speck Simulator (SpeckSim) from SnySense, available as a target in VLAB, differs slightly from other targets due to its spiking nature.
This tutorial assumes you are familiar with the basic concepts of spiking neural networks and the representation of spiking datasets.
It demonstrates how to use SpeckSim through the VLAB client API to run a model trained on MNIST, compute its accuracy, and analyze the spiking activity during inference (benchmarking type 2).
Preparing the environement
In a Python environment, install the daiedge-vlab and fill the setup.yaml file following the step-by-step user guide.
Install torch, torchvision packages.
Creating the model architecture
The model should be designed as a standard PyTorch model. It can then be converted to the spiking format required by SpeckSim using the Sinabs library.
VLAB handles the conversion from the PyTorch format to the spiking-compatible format. Therefore, to interface your model with SpeckSim through VLAB, provide it as a .pth file and ensure the input shape is specified as shown below.
SpeckSim currently supports only Linear and Conv2D layers and does not support bias terms.
import torch
import torch.nn as nn
model = nn.Sequential(
nn.Conv2d(1, 20, 5, 1, bias=False),
nn.ReLU(),
nn.AvgPool2d(2, 2),
nn.Conv2d(20, 32, 5, 1, bias=False),
nn.ReLU(),
nn.AvgPool2d(2, 2),
nn.Conv2d(32, 128, 3, 1, bias=False),
nn.ReLU(),
nn.AvgPool2d(2, 2),
nn.Flatten(),
nn.Linear(128, 500, bias=False),
nn.ReLU(),
nn.Linear(500, 10, bias=False),
)
torch.save({
"model": model,
"input_shape": (1, 28, 28),
}, "model.pth")Training the model
The model can be trained either before or after conversion to the spiking neural network (SNN) format.
In the current integration of SpeckSim within VLAB, the model is trained before conversion, using a standard (non-spiking) dataset. The conversion to the spiking format is then handled automatically by VLAB.
Alternatively, the model can be trained after conversion, directly as a spiking neural network using a spiking dataset. While this approach can yield better performance, it requires more advanced knowledge of training SNNs and therfore is not proposed by the VLAB.
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
data = torch.load("model.pth", weights_only=False)
model = data["model"]
input_shape = data["input_shape"]
# Define loss and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
# Prepare MNIST dataset
transform = transforms.Compose([transforms.ToTensor()])
train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)
# Training loop
epochs = 10
for epoch in range(epochs):
total_loss = 0
for images, labels in train_loader:
optimizer.zero_grad()
output = model(images)
loss = criterion(output, labels)
loss.backward()
optimizer.step()
total_loss += loss.item()
print(f"Epoch {epoch+1}/{epochs}, Loss: {total_loss/len(train_loader):.4f}")
# Save trained model
torch.save({
"model": model,
"input_shape": (1, 28, 28),
}, "model.pth")Creating spiking dataset
A spiking dataset can be generated from a standard dataset. Since parameters such as the time dimension can be adjusted, dataset creation is left to the user for maximum flexibility.
In this example, we use a time windows of size 5. At each time step, the probability of a spike occurring at pixel (x,y) is proportional to its normalized intensity value in the original image [0,1].
The optimal time window for encoding the data depends on the model architecture. An inappropriate time window can result in the absence of spikes at the model’s output during inference.
from torchvision import datasets
import numpy as np
WINDOWS = 5
test_dataset = datasets.MNIST(root='./data', train=False, download=True)
subset_data = test_dataset.data[:20]
subset_target = test_dataset.targets[:20]
subset_data = subset_data / 255
events = []
for i in range(len(subset_data)):
events.append([])
for t in range(WINDOWS):
image = subset_data[i]
for x in range(len(image)):
for y in range(len(image[x])):
if image[x][y] > np.random.rand():
event = (t,0,x,y) # (timestep, polarity, x coordinate, y coordinate)
events[i].append(event)
subset_targets = subset_target.tolist()
events_obj_array = np.array(events, dtype=object)
np.save("events.npy", events_obj_array, allow_pickle=True)
np.save("targets.npy", subset_targets)Benchmarking the model using VLAB
The Torch model trained on the MNIST dataset in a non-spiking format can be submitted to VLAB along with the spiking version of the MNIST inference dataset.
VLAB handles the conversion of the Torch model into a spiking neural network format and runs inference using SpeckSim.
While SpeckSim does not provide latency or energy consumption metrics, it does return model outputs that enable accuracy evaluation, along with detailed monitoring of spiking activity.
Monitoring the spiking activity helps verify that the architecture is functioning correctly, and it can also be used to estimate the computational load required by the device to perform inference. For guidance on how to interpret spiking activity, refer to the Sinabs documentation.
from daiedge_vlab import dAIEdgeVLabAPI
import numpy as np
import io
import json
# Authentify to the dAIEdge-VLab
api = dAIEdgeVLabAPI("setup.yaml")
dataset_name = api.uploadDataset("events.npy")
# Start a benchmark for a given target, runtime and model
benchmark_id = api.startBenchmark(
target = "specksim",
runtime = "pth",
model_path = "model.pth",
dataset = "events.npy"
)
print("benchmark ID:", benchmark_id)
# Blocking method - wait for the results
result = api.waitBenchmarkResult(benchmark_id)Computing accuracy
The model outputs spike events as a list of tuples. Each tuple represents a spike in the form (x, y, t, p), where t is the time step at which the spike occurs, and p is the index of the neuron that fired.
The output layer consists of 10 neurons, each corresponding to a class in the MNIST dataset. The predicted class is determined by identifying the neuron that fired the most during the inference window.
from daiedge_vlab import dAIEdgeVLabAPI
import io
BENCHAMRK_ID = # Fill out your benchmark ID
# Authentify to the dAIEdge-VLab
api = dAIEdgeVLabAPI("setup.yaml")
result = api.getBenchmarkResult(BENCHMARK_ID)
# Load the raw_outputs of the model
raw_bytes = result["raw_output"]
buffer = io.BytesIO(raw_bytes)
data = np.load(buffer, allow_pickle=True)
targets = np.load("targets.npy")
# Compute accuracy
success = 0
total = 0
for i, output_spikes in enumerate(data):
output_channels = output_spikes["p"]
prediction = np.argmax(np.bincount(output_channels))
print(f"Prediction: {prediction}, target: {targets[i]}")
if prediction == targets[i]:
success += 1
total += 1
print("Model accuracy: ", success/total)Monitoring spiking activity
Spiking activity is monitored during inference. Each spike event occurring in each layer is recorded as a list of events, where each event is represented as a tuple (x, y, t, p). Here, (x, y) identifies the source neuron that fired, t is the time step when the spike occurred, and p is the index of the target neuron that received the spike and performed a synaptic operation (SynOp).
By plotting the number of events per layer at each time step, one can assess whether the spiking activity remains stable throughout the time window. Additionally, counting the total number of events provides an estimate of the computational effort required for a single inference.
from daiedge_vlab import dAIEdgeVLabAPI
import matplotlib.pyplot as plt
from collections import Counter
# Authentify to the dAIEdge-VLab
api = dAIEdgeVLabAPI("setup.yaml")
result = api.getBenchmarkResult(6729)
report = result["report"]
total_spiking_activity = report["total_spiking_activity"]
inference_0_spiking_activity = total_spiking_activity[0]
# Step 1: Gather counts per timestep for each layer
layer_counts = {}
all_timesteps = set()
for layer, events in inference_0_spiking_activity.items():
counter = Counter()
for (_, _, t, _) in events:
counter[t] += 1
all_timesteps.add(t)
layer_counts[layer] = counter
# Step 2: Sort timesteps
all_timesteps = sorted(all_timesteps)
# Step 3: Build stacked bar data
bottom = [0] * len(all_timesteps)
fig, ax = plt.subplots()
for layer in sorted(layer_counts.keys()):
counts = [layer_counts[layer].get(t, 0) for t in all_timesteps]
ax.bar(all_timesteps, counts, bottom=bottom, label=f"Layer {layer}")
# Update bottom for stacking
bottom = [b + c for b, c in zip(bottom, counts)]
ax.set_xlabel("Timestep (t)")
ax.set_ylabel("Number of events")
ax.set_title("Stacked Event Count per Timestep")
ax.legend()
plt.savefig("spiking_activity.png")The figure below shows the spiking activity during the first inference of our trained model on the MNIST dataset. It demonstrates that the activity remains stable across the time window, with a consistent number of spikes occurring in each layer. In contrast, an untrained model would produce a different pattern, often with irregular or minimal spiking activity.
