API Reference¶
Auto-generated from docstrings via mkdocstrings. All public names are
re-exported at the package top level via the lazy-attr table in
chumicro_deploy/__init__.py; the per-module sections below mirror
the internal layout for readers who want to navigate by source file.
Device¶
chumicro_deploy.device
¶
Device: transport-agnostic configuration for a target board.
A :class:Device bundles the runtime identity (circuitpython /
micropython), the host-side connection details (serial address,
baudrate, CIRCUITPY drive path), and the deploy-mode preference.
:meth:Device.create_transport returns a concrete
:class:~chumicro_deploy.protocol.TransportProtocol instance, so
callers don't have to branch on runtime themselves.
Two fields, entrypoint_name and resource_prefix, aren't
read by :meth:create_transport. They are deploy-config carried
on Device so a deploy takes one configured object.
Device
dataclass
¶
Configuration for a target board.
Constructed explicitly in code, from a dict
(Device.from_dict(...)), or via the built-in
devices.yml loader
(chumicro_deploy.config.default.load_devices_yml), or a
third-party loader registered through the
chumicro_deploy.config_loaders entry-point group.
Attributes:
| Name | Type | Description |
|---|---|---|
transport |
str
|
Runtime identifier, |
address |
str
|
Serial port path ( |
baudrate |
int
|
Serial baudrate. Only meaningful for the
CircuitPython transport; the MicroPython transport uses
|
deploy_mode |
str
|
|
entrypoint_name |
str | None
|
Top-level script the runtime executes on
boot. Defaults vary per runtime ( |
resource_prefix |
str
|
On-device directory where library files land at deploy time. |
transport_factory |
Callable[[Device], TransportProtocol] | None
|
Override hook for :meth: |
The CIRCUITPY drive (CP flash-mode deploys) is auto-resolved at
deploy time by scanning the host's mount points and matching the
connected board's UID / machine string against each candidate's
boot_out.txt, so no per-device drive path is configured here.
effective_entrypoint
property
¶
Return the entrypoint filename, resolving runtime default when unset.
CircuitPython boards boot code.py; MicroPython boards boot
main.py. Override via :attr:entrypoint_name when the
target runs something else.
from_dict(data)
classmethod
¶
Construct a :class:Device from a mapping of field names.
Accepts the same keys as the constructor: transport,
address, and optional baudrate, deploy_mode,
entrypoint_name, and resource_prefix. Unknown keys
are ignored so YAML / TOML / JSON inputs with extra metadata
fields (id, description, circuitpy_drive_path,
etc.) pass through without filtering.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Mapping[str, Any]
|
Mapping of field names to values. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
Missing |
create_transport()
¶
Construct the concrete transport for this device.
Returns a :class:~chumicro_deploy.micropython_transport.MicropythonTransport
or :class:~chumicro_deploy.circuitpython_transport.CircuitpythonTransport
depending on :attr:transport. Deploy-mode translation
(ram/flash to the transport's native label) happens here.
When :attr:transport_factory is set, returns its result
instead.
Deployer¶
chumicro_deploy.deployer
¶
Deployer: orchestrate file staging + entrypoint execution on a Device.
The :class:Deployer owns the end-to-end "push code onto a board and
run it" flow. It consumes a :class:~chumicro_deploy.sources.FileSource,
constructs a transport via :class:~chumicro_deploy.device.Device,
stages every file the source returns, executes the entrypoint, and
returns a :class:~chumicro_deploy.result.DeployResult.
The transport-level primitive this builds on is
:meth:~chumicro_deploy.protocol.TransportProtocol.deploy_files.
Per-file iteration and per-group resets are exposed through the
richer stage() / execute() flow instead.
Before each deploy, a pre-flight pass auto-promotes a RAM-mode device
to flash when the source ships a library marked
[tool.chumicro] requires_flash = true. The promotion is announced
through on_preflight_message (stderr by default). Pass
force_deploy_mode to bypass pre-flight.
Deployer
¶
End-to-end deploy orchestrator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device
|
Device
|
Target board configuration. :attr: |
required |
device
property
¶
The target :class:Device this Deployer was constructed with.
deploy_diff(source, *, clean=True, wipe=False, force_deploy_mode=None, on_progress=None, on_file_staged=None, on_file_deleted=None, on_execute_line=None, on_preflight_message=None, tail_seconds=None)
¶
Diff-deploy source: delete stale in-scope files, then deploy.
- Connect.
- Ask the transport for every in-scope file currently on the
device (
list_files_in_scope). - Compute the stale set, the paths on the device that aren't in the new payload.
- Delete the stale set (
delete_files). - Hand off to the normal :meth:
deploy_filesfor the actual write + execute.
Out-of-scope files (user-uploaded images, hand-edited
settings.toml, etc.) are never touched; see
:func:chumicro_deploy.protocol.is_in_deploy_scope for the rule.
Mode-aware: in RAM-mode deploys (CP RAM, MP mount) the
transport's list_files_in_scope returns an empty list and
the diff routine collapses to a plain :meth:deploy_files
call, because RAM mode never wrote to flash so there's nothing
persistent to diff against.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
FileSource
|
:class: |
required |
clean
|
bool
|
Clean-slate reconcile (the default). The diff
scope is the whole device minus the closed keep set
(:data: |
True
|
wipe
|
bool
|
When |
False
|
force_deploy_mode
|
str | None
|
Override the pre-flight requires_flash
policy. |
None
|
on_progress
|
Callable[[float, str], None] | None
|
Optional |
None
|
on_file_staged
|
Callable[[str], None] | None
|
Forwarded to |
None
|
on_file_deleted
|
Callable[[str], None] | None
|
Per-file callback invoked with each stale on-device path before deletion. |
None
|
on_execute_line
|
Callable[[str], None] | None
|
Forwarded to |
None
|
on_preflight_message
|
Callable[[str], None] | None
|
Optional callback for the
"switching to flash mode" message the requires_flash
pre-flight emits. Defaults to |
None
|
tail_seconds
|
float | None
|
CP-only override for how long the transport
captures serial output after the entrypoint's
soft-reboot. |
None
|
Returns:
| Type | Description |
|---|---|
DeployResult
|
class: |
DeployResult
|
|
DeployResult
|
|
DeployResult
|
(the deletion list is observable via on_file_deleted |
DeployResult
|
during the call but not retained on the result). |
Result types¶
chumicro_deploy.result
¶
DeployResult and DeployError: outcome types returned by the deployer.
Defined in their own module so callers can annotate against a stable return type without pulling in the orchestration code.
DeployError
¶
Bases: Exception
A deploy step failed in a way the caller should surface directly.
Raised for deploy-specific failures (oversized payload, missing
entrypoint, transport staging error after retries exhausted).
Transport-level errors propagate as their own types (e.g.
:class:~chumicro_deploy.circuitpython_transport.CircuitpythonTransportError)
so callers can branch on the underlying cause.
DeployResult
dataclass
¶
Outcome of a single Deployer.deploy_diff() call.
Attributes:
| Name | Type | Description |
|---|---|---|
success |
bool
|
|
staged_files |
list[str]
|
On-device paths that were written (for flash mode) or the host-side staging paths that fed the RAM-mode inline execution. Empty when staging never completed. |
execute_output |
str
|
Combined stdout captured from the board while the entrypoint ran. Empty when the execute step never fired. |
traceback |
str | None
|
Traceback text extracted from |
File sources¶
chumicro_deploy.sources
¶
File sources: plug-in input types that feed the Deployer.
A :class:FileSource produces two things: a mapping of on-device
paths to byte contents, and the name of the entrypoint file the
runtime should boot into. Three built-ins cover the common cases:
- :class:
FileMapSourcewhen the caller already has an in-memory dict. - :class:
DirectorySourceto ship a directory tree from disk. - :class:
ImportGraphSourceto walk imports from an entrypoint file and ship exactly the transitively-reachable modules.
Custom sources implement the :class:FileSource protocol. The
protocol is :func:~typing.runtime_checkable so isinstance works
at the deployer boundary.
UnresolvedImportError
¶
Bases: Exception
Raised when an import walk reaches a module that ships nowhere.
:class:ImportGraphSource refuses the deploy at construction time when an
import in the bundle resolves to no file under any search path and the
module name is not a known device-runtime built-in (see
:data:chumicro_deploy.import_allowlist.DEVICE_BUILTIN_MODULES). Such an
import would ship silently and raise :exc:ImportError from the device
runtime at first boot; refusing here turns a boot-time crash into a
deploy-time error that names the importing file and the missing module.
The message leads with "Deploy refused: unresolved import" so
:func:chumicro_deploy.recovery.classify_deploy_failure routes it to
:attr:~chumicro_deploy.recovery_kind.DeployFailureKind.UNRESOLVED_IMPORT
without coupling the classifier to this subclass.
Attributes:
| Name | Type | Description |
|---|---|---|
unresolved |
Every |
FileSource
¶
Bases: Protocol
What the Deployer expects from any file source.
Implementations return the deploy-time file map and declare which of those files is the entrypoint. The Deployer does not mutate either result. Sources own their own state.
files()
¶
Return on-device path → file bytes.
Keys are on-device paths (typically starting with /) that
the Deployer writes onto the target. Values are the raw file
contents.
entrypoint()
¶
Return the on-device entrypoint path.
Must be one of the keys returned by :meth:files. The
Deployer executes this file after staging.
FileMapSource
¶
A source backed by an in-memory dict.
Strings in the input are encoded as UTF-8. Binary values pass through unchanged.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
files
|
dict[str, str | bytes]
|
Mapping of on-device paths to file contents. Values
may be |
required |
entrypoint
|
str
|
Key in files that is the boot file. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If entrypoint is not a key of files. |
DirectorySource
¶
A source that ships every file under a host-side directory.
Walks root recursively and reads every file as bytes. The
on-device path is the file path relative to root, joined with
resource_prefix (default /).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
root
|
Path
|
Host directory whose contents are deployed. |
required |
entrypoint
|
str
|
On-device entrypoint path. Must end up as a key in the produced file map, i.e. must be a file under root (relative to resource_prefix). |
required |
resource_prefix
|
str
|
On-device prefix prepended to each file's
path. Defaults to |
'/'
|
excluded_names
|
frozenset[str] | None
|
Filename / directory names to skip entirely
(exact match, not glob). Defaults to common artifacts
( |
None
|
target_runtime
|
str | None
|
When set ( |
None
|
Raises:
| Type | Description |
|---|---|
NotADirectoryError
|
If root does not exist or is not a directory. |
ValueError
|
If entrypoint is not produced by the walk. |
ImportGraphSource
¶
A source that walks Python imports starting from an entrypoint.
Parses entrypoint with the ast module, collects every
import and from ... import target, resolves each one
against search_paths (first match wins), and recursively walks
the resolved modules. Packages are walked through their
__init__.py. Dynamic imports (importlib.import_module,
__import__) are not detected; pass those names explicitly via
extra_modules.
A module name that doesn't resolve against any search path is
either a device-runtime built-in (gc, time, board)
the host can't provide, or a library the walk should have found
but didn't (missing from the search paths, or a typo). The two
look identical at the AST level, since both are bare module names.
:class:ImportGraphSource distinguishes them by an explicit
allowlist: a name on
:data:chumicro_deploy.import_allowlist.DEVICE_BUILTIN_MODULES
is skipped as a built-in; any other unresolved required import
is collected, and the constructor raises
:class:UnresolvedImportError naming every importing file and
missing module rather than shipping an import that would
:exc:ImportError at first boot. Two cases are exempt: an
import guarded by try: ... except ImportError (a deliberate
optional fallback), and the speculative module.name probe the
walk uses to resolve from module import name against a real
submodule. When name is a function or class in
module/__init__.py instead, the probe doesn't resolve and is
silently dropped.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entrypoint
|
Path
|
Host path to the entrypoint file. Deployed as
|
required |
search_paths
|
list[Path]
|
Directories searched in order when resolving
imports. Typical chumicro usage supplies |
required |
extra_modules
|
list[str] | None
|
Dotted module names to force-include even when
|
None
|
device_entrypoint
|
str
|
On-device path for the entrypoint (default
|
'/code.py'
|
resource_prefix
|
str
|
On-device prefix for every non-entrypoint
module file (default |
'/lib'
|
target_runtime
|
str | None
|
When set ( |
None
|
Honors a module-level __chumicro_skip_factories__ opt-out on
entrypoint; see :mod:chumicro_deploy.skip_factories for the
marker shape and matching rules. Diagnostics (direct-import
overrides, dead-skip notices) surface through
:attr:skip_factories_warnings.
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If entrypoint does not exist. |
NotADirectoryError
|
If any of search_paths is not a directory. |
ValueError
|
If |
UnresolvedImportError
|
If a required import resolves to no
file under any search path and is not a known device
built-in. Carries every offending |
skip_factories_warnings()
¶
Return diagnostic messages emitted by the skip mechanism.
Direct-import overrides and dead-skip notices land here. Empty tuple when the deploy used no skip mechanism or nothing was worth saying.
host_paths()
¶
Return the host filesystem paths every contributed module was read from, including the entrypoint.
Returns a copy so callers can mutate without affecting the source's internal record.
unresolved_imports()
¶
Return every (importing_file, module_name) the walk couldn't ship.
A pair is collected for each required import that resolves to no file
under any search path and isn't a known device built-in. A
successfully-constructed :class:ImportGraphSource always returns an
empty list: a non-empty set aborts construction with
:class:UnresolvedImportError (whose unresolved attribute carries
the same pairs). The accessor exists so a caller that builds the
source under its own try can read the collected pairs off either
side.
Probe¶
chumicro_deploy.probe
¶
probe_device: one-shot runtime-identity probe over a transport.
Thin wrapper over :meth:TransportProtocol.probe_implementation that
owns the transport lifecycle (connect / probe / disconnect) and
returns a :class:DeviceInfo callers can surface to users.
CPU UID is populated from the probe script itself.
:data:~chumicro_deploy.protocol.PROBE_IMPLEMENTATION_SCRIPT reads
microcontroller.cpu.uid on CircuitPython and
machine.unique_id() on MicroPython and emits it as the fourth
field of the __CHU_IMPL__: marker line.
DeviceInfo
dataclass
¶
What :func:probe_device gathered from a connected board.
Attributes:
| Name | Type | Description |
|---|---|---|
implementation |
DeviceImplementation | None
|
Parsed |
board_id |
str
|
Normalized board identifier (e.g.
|
uid |
str
|
Hex-uppercase CPU UID probed from
|
probe_device(device)
¶
Connect, probe, disconnect, and return what the board reports.
Exceptions during the probe propagate. Wrap the call when a soft-failure shape is needed. The transport is always disconnected, even on error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
device
|
Device
|
Target :class: |
required |
Returns:
| Type | Description |
|---|---|
DeviceInfo
|
class: |
DeviceInfo
|
when the board returned the probe marker. |
DeviceInfo
|
|
Firmware¶
chumicro_deploy.firmware
¶
Firmware URL resolution + flashing.
Two surfaces: :func:resolve_firmware_url (pure URL formatter) and
:func:flash_firmware (download + apply, destructive). See
:func:flash_firmware's docstring for the UF2-vs-esptool method
selection and per-path recovery strategies.
FlashFirmwareError
¶
Bases: Exception
Raised when a flash step fails.
Message always includes recovery guidance, e.g. "put the board in bootloader mode and retry" or "install esptool first". Catchers typically surface the message directly to the user rather than trying to introspect.
resolve_firmware_url(board_id, runtime, version, *, language=_DEFAULT_LANGUAGE)
¶
Format the firmware download URL for a known board + version.
Pure URL formatter. No network access. Use this when you have
an explicit version (e.g. from CI, a release script, or
user-supplied). For "give me the latest version available" use
:func:firmware_url.derive_firmware_url or
:func:firmware_url.latest_circuitpython_url instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
board_id
|
str
|
Board identifier. CircuitPython boards use the
Adafruit ID (e.g. |
required |
runtime
|
str
|
|
required |
version
|
str
|
Firmware version. For CircuitPython, the Adafruit
release label ( |
required |
language
|
str
|
Adafruit language code (CircuitPython only).
Defaults to |
_DEFAULT_LANGUAGE
|
Returns:
| Type | Description |
|---|---|
str
|
Fully-formed download URL. |
Raises:
| Type | Description |
|---|---|
UnresolvedFirmwareError
|
If runtime is not supported, or
if any required field is empty. |
flash_firmware(url, device, *, reflash_method=None, bootloader_drive_path=None, interactive=True, erase_flash=True, flash_offset='0x0', on_progress=None)
¶
Download url and flash it onto device.
Destructive: overwrites whatever firmware is currently installed. Progress is reported in rough halves, 0.0 to 0.5 covers download and 0.5 to 1.0 covers flash.
Method selection:
"uf2"for RP2040 / RP2350 (Pi Pico family) and any board shipping TinyUF2. Requires a.uf2URL and writes through the UF2 bootloader drive. Programmatic bootloader entry works on CircuitPython and on MicroPython ports that implementmachine.bootloader()."esptool"for ESP32 family boards (ESP32, S2, S3, C3, C6) regardless of runtime. Requires a.binURL. The caller puts the board in ROM bootloader (the helper handles programmatic entry on boards wired for USB-CDC DTR/RTS, and prompts for a manual BOOT-hold otherwise).
When reflash_method is None, the method is inferred from
the URL extension (.uf2 to UF2, .bin to esptool).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
str
|
Firmware download URL, typically from
:func: |
required |
device
|
Device
|
Target :class: |
required |
reflash_method
|
str | None
|
|
None
|
bootloader_drive_path
|
Path | None
|
UF2 path only. Skips auto-detection and writes directly to this path. |
None
|
interactive
|
bool
|
UF2 / esptool path. When |
True
|
erase_flash
|
bool
|
esptool path only. When |
True
|
flash_offset
|
str
|
esptool path only. Defaults to |
'0x0'
|
on_progress
|
Callable[[float, str], None] | None
|
Optional |
None
|
Raises:
| Type | Description |
|---|---|
FlashFirmwareError
|
Download, bootloader entry, drive detection, copy, reboot, or esptool invocation failed. The message names the step and carries recovery guidance. |
ValueError
|
Unknown reflash_method, or reflash_method is
|
Interactive recovery¶
chumicro_deploy.recovery
¶
Recovery layer around :class:Deployer.
Transport failures during deploy fall into a small set of classes
(port busy, CIRCUITPY drive missing, raw REPL unresponsive, rsync
balked mid-copy). Most are recoverable when the user takes a
concrete physical action (close the program holding the port, tap
RESET, replug USB) and retries. This module classifies a transport
exception into a :class:DeployFailureKind, looks up the canned
:class:RecoveryPlan for that kind, and wraps :class:Deployer in
a coaching loop that surfaces the plan and retries when retryable.
PortHolder
dataclass
¶
A process currently holding a serial port open.
Returned by :func:diagnose_port_holders so output to the user
can name the process blocking the deploy instead of the generic
"close the app holding the port" hint.
Attributes:
| Name | Type | Description |
|---|---|---|
pid |
int
|
Process id reported by |
command |
str
|
Full command line via |
RecoveringDeployer
¶
:class:Deployer wrapper that classifies failures and coaches recovery.
Two modes selected by the prompt argument:
- Non-interactive (
prompt=None, the default): runs once, prints the classified failure + ordered fix steps on a transport error, then re-raises. - Interactive (
prompt=inputor any callable taking the prompt text and returning the user's reply): runs up to max_attempts times, asks between attempts. An empty / pure- whitespace reply continues; any reply starting withq,a, oreaborts.
On a :class:DeployResult with success=False and a
traceback, both modes print the traceback + the
TRACEBACK_RETURNED plan and return the result unchanged. A
source-level bug isn't something retrying the same bytes will fix.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
deployer
|
Deployer
|
Underlying :class: |
required |
prompt
|
Callable[[str], str] | None
|
Callable that asks the user whether to retry. |
None
|
max_attempts
|
int
|
Ceiling on retry attempts when prompt is set. Ignored in non-interactive mode (always 1 attempt). |
_DEFAULT_MAX_ATTEMPTS
|
output
|
Callable[[str], None]
|
Injectable output sink. Defaults to :func: |
print
|
fskit_wedge_detector
|
Callable[[], bool]
|
Injectable probe for the macOS FSKit
wedge. Promotes |
detect_fskit_wedge
|
deployer
property
¶
The underlying non-recovery :class:Deployer.
deploy_diff(source, *, clean=True, wipe=False, on_progress=None, on_file_staged=None, on_file_deleted=None, on_execute_line=None, tail_seconds=None, force_deploy_mode=None)
¶
Diff-deploy source with classify / coach / (optional retry).
classify_deploy_failure(error)
¶
Map a deploy-path exception to a :class:DeployFailureKind.
The classifier is intentionally string-based: it inspects
str(error).lower() against :data:_CLASSIFICATION_TABLE.
That keeps the classifier decoupled from the specific exception
subclass, which matters because a raised
CircuitpythonTransportError often wraps a SerialException
or OSError whose text is the real signal.
Returns :attr:DeployFailureKind.UNKNOWN when no row matches.
diagnose_port_holders(port_path, *, runner=subprocess.run)
¶
Return processes currently holding port_path open.
Uses lsof -F pcn <port> on POSIX. The output format is one
field per line tagged with a single character (p for PID,
c for short command, n for path). Each PID record
starts with a p line, and subsequent c / n lines belong
to it until the next p or EOF.
Returns an empty list on Windows (no portable equivalent
without handle.exe), when lsof isn't installed, when
the port is not held, or on any subprocess failure. The
caller treats absence of holders as "we couldn't tell, fall
through to the generic recovery hint" rather than an error.
A deploy interrupted by Ctrl-C that leaves an orphan
chumicro-deploy / mpremote subprocess holding the port surfaces
as "failed to access ... it may be in use by another program".
Showing the PID lets the user kill the right thing without
running lsof themselves.
recovery_plan_for(kind)
¶
Return the canned :class:RecoveryPlan for kind.
macOS FSKit wedge detection¶
chumicro_deploy.macos_fskit
¶
macOS FSKit / DiskArbitration wedge detection.
Recent macOS releases moved the FAT (msdosfs) driver out of the
kernel and into a user-space FSKit extension
(com.apple.fskit.msdos.appex). When that extension hits an
internal error mid-probe (most often on small CIRCUITPY FAT12
volumes), it can leave diskarbitrationd stuck in an
uninterruptible kernel wait. Symptoms:
diskutil listhangs indefinitely.- Newly inserted CIRCUITPY drives never appear under
/Volumes. - Unplug / replug of the board does nothing; the bus enumerates
fine (
system_profiler SPUSBDataTypeshows the device) but no mount ever happens.
The cheapest reliable signal is the state column of
diskarbitrationd: a healthy daemon sits in Ss (interruptible
sleep), a wedged one sits in Us / U* (uninterruptible wait).
:func:detect_fskit_wedge reads that state via pgrep + ps
and reports the wedge.
Recovery requires sudo (killing system daemons), so this module
does not auto-run it. It just surfaces the exact command in
:data:MACOS_FSKIT_RECOVERY_COMMAND for the user-facing coaching
output to paste verbatim.
detect_fskit_wedge(*, runner=subprocess.run, platform=_sys_module.platform)
¶
Return True when macOS diskarbitrationd is wedged.
Always returns False on non-macOS platforms, because the FSKit
rewrite is Apple-only, so the wedge mode does not exist
elsewhere.
Implementation:
pgrep diskarbitrationdto find the PID. No PID (daemon missing or in the process of respawning) means not wedged.ps -o state= -p <pid>reads the process state column. A state containingUmeans uninterruptible kernel wait, which is the wedge signature. Healthy daemons sit inS(interruptible sleep).
Both subprocess calls get short timeouts, because a wedged system can stall other commands too, and we don't want detection itself to hang.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
runner
|
Callable[..., CompletedProcess]
|
Injectable subprocess runner. Must match the
|
run
|
platform
|
str
|
Injectable |
platform
|
Returns:
| Type | Description |
|---|---|
bool
|
|
bool
|
|
bool
|
timeout, non-zero exit) is treated as "not wedged" so a |
bool
|
false positive can never block a legitimate retry. |
Host platform compatibility¶
chumicro_deploy.host_platform
¶
Host-platform compatibility checks for chumicro-deploy.
The deploy package targets developer laptops: fully supported on
macOS, fully viable on Linux, and explicitly unsupported on native
Windows. This module surfaces those gaps as fast-failing errors with
actionable remediation hints rather than letting them manifest as
no devices found deep inside the deploy flow.
Two checks live here:
- :func:
check_supported_platform, called before any path that enumerates serial ports or USB mount points, so Windows users see a "use WSL2" message immediately instead of an empty-port-list dead end. - :func:
check_rsync_available, called before flash-mode deploy. The CircuitPython flash transport shells out torsyncand there is no fallback (FAT32 timestamp semantics makeshutil.copytreeunreliable for incremental writes). Detecting absence up-front beats failing partway through staging.
WindowsNotSupportedError
¶
Bases: RuntimeError
Raised when chumicro-deploy is invoked on native Windows.
RsyncMissingError
¶
Bases: RuntimeError
Raised when rsync is required for flash-mode deploy but
is not on PATH.
is_native_windows()
¶
Return True on native Windows (win32 / cygwin).
WSL2 reports sys.platform == "linux" and is treated as Linux
throughout the deploy code, which is correct since the POSIX
paths work inside WSL2.
check_supported_platform()
¶
Raise :class:WindowsNotSupportedError on native Windows.
Call before any path that enumerates serial ports or USB mount points, so users see one clean error before the deploy flow tries to walk POSIX-only paths. No-op on macOS, Linux, and WSL2.
install_hint_for_rsync()
¶
Return a platform-specific install command for rsync.
Detects common Linux package managers via :func:shutil.which
and produces a copy-paste-able sudo <manager> install rsync
line. Falls back to a generic message when no recognized
manager is on PATH. On macOS the system ships rsync, so the
hint points at Xcode Command Line Tools / Homebrew as the
recovery path.
check_rsync_available()
¶
Raise :class:RsyncMissingError if rsync is not on PATH.
Call before a CircuitPython flash-mode deploy so a missing rsync
surfaces as one clear error before the transport opens a serial
connection and starts staging. The error embeds the install hint
from :func:install_hint_for_rsync.
Devices.yml schema and loader registry¶
The chumicro_deploy.config package owns the devices.yml schema.
load_devices_yml is the built-in loader (registered under the
"default" entry-point name); third parties register their own
config formats via the chumicro_deploy.config_loaders entry-point
group, and discover_config_loaders collects every registered
loader keyed by name.
chumicro_deploy.config
¶
Opt-in configuration loaders for :class:~chumicro_deploy.Device.
chumicro-deploy ships one built-in loader (registered as
"default") that reads the devices.yml schema defined in
:mod:chumicro_deploy.config.default. Third parties with a
different config format register a loader by declaring a Python
entry point in the chumicro_deploy.config_loaders group. A
template repo's pyproject.toml looks like:
.. code-block:: toml
[project.entry-points."chumicro_deploy.config_loaders"]
myformat = "my_pkg.loader:load"
Where my_pkg.loader.load(path, *, device_id=None) returns a
:class:Device.
:func:discover_config_loaders collects the built-in default
loader plus every registered third-party loader into a single
{name: callable} mapping that downstream code can dispatch from.
discover_config_loaders()
¶
Return the registered config-loader callables keyed by name.
The default entry is always present (built-in). Third-party
entries come from Python entry points in the
chumicro_deploy.config_loaders group. Any name collision
with default is rejected by the registry so built-in
behavior is not silently shadowed.
Returns:
| Type | Description |
|---|---|
dict[str, ConfigLoader]
|
|
dict[str, ConfigLoader]
|
returns a :class: |
chumicro_deploy.config.default
¶
Default YAML config loader that ships with chumicro-deploy.
Loads the devices.yml schema that chumicro-deploy defines and
owns. Any project (its own workspace, a project-workspace template,
or a third-party consumer) can write this shape to configure its
deploy targets without depending on any upstream tooling.
The schema (stable subset that the loader accepts):
.. code-block:: yaml
defaults:
micropython: <id of default MP device>
circuitpython: <id of default CP device>
deploy_mode: ram
devices:
- id: my-pico-w-mp
runtime: micropython # → Device.transport
address: /dev/cu.usbmodem213101
serial_baudrate: 115200
deploy_mode: ram # optional; falls back to defaults.deploy_mode
# description and other keys are tolerated
# but ignored by this loader.
Required fields per device: id, runtime, address.
Everything else is optional. Extra keys are silently ignored so
consumers that carry additional metadata (test-orchestration hints,
setup commands, descriptions) can share the same file.
Importing this module pulls in PyYAML (already a dependency of
:mod:chumicro-deploy), so the import cost is paid only when
devices.yml is actually used.
Third parties that need a different shape (JSON, TOML, a custom
YAML layout) can register their own loader via the
chumicro_deploy.config_loaders entry-point group; see
:func:chumicro_deploy.config.discover_config_loaders.
DeviceConfigError
¶
Bases: Exception
Raised when a devices.yml is missing or fails validation.
DeviceEntry
dataclass
¶
A single device from a validated devices.yml registry.
DeviceDefaults
dataclass
¶
Top-level defaults from the defaults: section of devices.yml.
Controls which devices are targeted from IDE play buttons, the default deploy mode for all devices, and which runtimes the IDE targets.
load_device_registry(path=None, *, workspace_root=None)
¶
Load and validate devices.yml into (entries, defaults).
Each entry is a :class:DeviceEntry carrying the description and
any extra keys from the YAML alongside the deploy-relevant fields.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path | None
|
Explicit path to |
None
|
workspace_root
|
Path | None
|
Directory that contains |
None
|
Returns:
| Type | Description |
|---|---|
tuple[list[DeviceEntry], DeviceDefaults]
|
Tuple of |
Raises:
| Type | Description |
|---|---|
DeviceConfigError
|
File missing, malformed, or fails validation. |
load_devices(path=None, *, workspace_root=None)
¶
Load just the validated device list from devices.yml.
Convenience wrapper around :func:load_device_registry.
resolve_ide_devices(devices, defaults)
¶
Return the devices an IDE play button should target.
For each runtime selected by defaults.ide_runtime, picks the
device whose ID matches defaults.<runtime>, or the first
device of that runtime when no default is set.
load_raw_entries(path)
¶
Parse a devices.yml into raw entries + defaults dict.
Reads the YAML and returns the devices: list and the
defaults: mapping verbatim: no field validation, no
:class:Device construction, no normalization. Richer schemas
layer their own validation on top of this, so the on-disk shape
is defined in one place.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path | str
|
Filesystem path to the YAML file. |
required |
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]]
|
Tuple of |
dict[str, Any]
|
|
tuple[list[dict[str, Any]], dict[str, Any]]
|
missing). defaults is the |
tuple[list[dict[str, Any]], dict[str, Any]]
|
(empty dict when missing). |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
The YAML file does not exist. |
ValueError
|
The YAML root is not a mapping, or
|
load_devices_yml(path, *, device_id=None, runtime=None)
¶
Load one device from a devices.yml file.
Resolution precedence:
- device_id wins outright.
- runtime picks
defaults.<runtime>when device_id isNone, so a single-runtime caller disambiguates a two-runtimedefaults:block without naming the device id. - Single-default fallback: when both device_id and runtime
are
None, exactly one runtime default in the file picks itself, otherwise raises.
device_id and runtime are mutually exclusive: passing both raises so the caller cannot accidentally override a specific id with a runtime hint.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path | str
|
Filesystem path to the YAML file. |
required |
device_id
|
str | None
|
Which entry to return. |
None
|
runtime
|
str | None
|
One of |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
Device
|
class: |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
The YAML file does not exist. |
ValueError
|
The file has no matching device, the
|
load_devices_yml_raw(path, *, device_id=None, runtime=None)
¶
Resolve one device and return its raw devices.yml entry.
Identical resolution to :func:load_devices_yml (same precedence,
same errors) but returns the unmapped entry dict, including
hardware and other keys the :class:Device dataclass drops.
Use when fields off the schema are needed alongside the transport
facade (firmware-URL derivation reads hardware.firmware_source).
Transport protocol¶
chumicro_deploy.protocol
¶
Explicit transport protocol for ChuMicro device transports.
Captures the duck-typed transport contract that
:class:CircuitpythonTransport and :class:MicropythonTransport
both satisfy, in a form type checkers can enforce.
Two protocols are defined:
- :class:
TransportProtocolis the minimum every transport must implement:connect,stage,execute,soft_reset,reset,recover,disconnect. - :class:
ExtendedTransportProtocoladds the CircuitPython-specific RAM-mode chunking helpers (execute_scripts,probe_free_memory,inline_script_budget_bytes).MicropythonTransportdoes not need these, because there is no per-script RAM budget on mpremote.
CPython-only: these protocols ride on typing.Protocol, which only
the workbench tooling uses (the cross-runtime device libraries that
satisfy the contract don't import this module).
Runtime
¶
Bases: StrEnum
Supported device runtime identifiers.
Values round-trip as plain strings so Device(transport="circuitpython")
and Device(transport=Runtime.CIRCUITPYTHON) are interchangeable.
CLI argparse choices, config-file loaders, and third-party callers
should reference members of this enum rather than string literals
to keep the allowed set in one place.
DeployMode
¶
Bases: StrEnum
User-facing deploy-mode preference.
RAM keeps edits off the board's flash (inline exec on
CircuitPython, mount on MicroPython). FLASH writes files
persistently (CIRCUITPY drive copy on CircuitPython, copy on
MicroPython). The transport-internal mount-label mapping
(ram → mount, flash → copy) is handled in
:meth:~chumicro_deploy.device.Device.create_transport.
ReflashMethod
¶
Bases: StrEnum
Firmware reflash backend selection.
UF2 drives the UF2 bootloader drive path (Pi Pico family,
TinyUF2 boards). ESPTOOL shells out to esptool over
serial for ESP32-family boards. See
:func:~chumicro_deploy.firmware.flash_firmware for the method
selection guide.
DeviceImplementation
dataclass
¶
Runtime identity probed from a connected board.
Populated by probe_implementation on transports that support
it. Identifies the runtime + board behind a connected device, and
pins a UID that disambiguates two boards of the same model, so a
mounted CIRCUITPY drive can be matched to its connected board
without a mount-order-dependent path.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
|
version |
str
|
Dotted version from |
machine |
str
|
|
uid |
str
|
Hex-uppercase CPU / module unique ID, sourced from
|
UnsupportedExtraFilesError
¶
Bases: NotImplementedError
Raised when transport.stage(extra_files=...) can't be honored.
Currently raised only by CircuitPython RAM mode: there is no
writable device-side filesystem to land bytes on (RAM mode runs
inline-execed source via raw REPL; CIRCUITPY is host-write but
a host write while the device is running can trigger a soft reset
that wipes the in-memory state). Staging an on-device file
artifact (typically runtime_config.msgpack for
chumicro_config.load_runtime_config()) requires flash mode.
TimeSource
¶
Bases: Protocol
Structural interface for an injectable time source.
FakeTime from :mod:chumicro_deploy.testing is one conforming
implementation that eliminates wall-clock waits.
DeviceTransportError
¶
Bases: Exception
Base for both runtimes' transport errors.
Orchestration that treats the transports uniformly (the
RecoveringDeployer retry loop, the failure classifier)
catches this instead of enumerating per-runtime subclasses, so
adding a runtime doesn't mean auditing every catch site.
MidDeployDisconnected
¶
Bases: DeviceTransportError
Base for the per-runtime "device dropped mid-deploy" errors.
A distinct branch so callers can except "the cable came out"
without conflating it with other transport errors. The original
:class:OSError is attached as :attr:cause so callers that
need the underlying errno can read it without re-parsing
:func:str of the wrapper.
TransportProtocol
¶
Bases: Protocol
Minimum transport contract every device transport must satisfy.
connect()
¶
Verify the device is reachable.
stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)
¶
Prepare the host-side staging area and (mode-dependent) push to device.
extra_modules are additional Python files that the test files
import as top-level modules. Each is registered as importable
on the device alongside library sources: in RAM mode they join
staged_sources so the inline bootstrap registers them; in
flash / copy / mount modes they land at the device root next
to the test files.
extra_files are non-Python files to land at named device paths.
Keys are absolute device paths ("/runtime_config.msgpack");
values are the bytes to write. Typical use is staging a
runtime_config.msgpack alongside test files so on-device
code can call :func:chumicro_config.load_runtime_config.
Per-mode semantics: flash and copy modes write
each file to the device's filesystem alongside library + test
sources. Mount mode writes to the host directory mounted as the
device filesystem. RAM mode raises
:class:UnsupportedExtraFilesError because it has no writable
device-side filesystem to land bytes on.
execute(bootstrap_script, *, on_line=None)
¶
Run bootstrap_script on the device and return captured stdout.
When on_line is provided, each captured stdout line is
dispatched to the callback as it arrives over the serial link,
before :meth:execute returns. Lines are dispatched without
their trailing newline; \r\n is normalized to \n.
When on_line is None (the default), behaviour is the
request/response shape every existing call site assumes, and
captured stdout still comes back as the return value.
Streaming dispatch lets a host-side consumer react mid-execute: for example, a test fixture that opens a TCP connection only once the board prints a "server ready" line, or a long-running bake harness that surfaces board output as it lands instead of after the bootstrap finishes.
run_script(script, *, timeout=10.0)
¶
Run script on the device without staging and return captured stdout.
Sibling of :meth:execute for short, self-contained scripts
that don't need any staged sources / test files / harness:
e.g. a feature-detection probe (try: import esp32 etc.),
a quick sys.implementation query, an inline diagnostic.
Bypasses the stage()-must-be-called-first precondition
that :meth:execute enforces.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
script
|
str
|
Self-contained Python source to run via the raw REPL. No imports of staged modules, only built-ins and runtime-provided modules. |
required |
timeout
|
float
|
Idle-timeout for the script's output, in seconds. |
10.0
|
Returns:
| Type | Description |
|---|---|
str
|
Captured stdout from the device. |
soft_reset()
¶
Soft-reset the interpreter to clear modules and free heap.
recover()
¶
Aggressive reset after a failure when board state is unknown.
disconnect()
¶
Release transport resources (serial port, mounts, staging dir).
probe_implementation()
¶
Query sys.implementation on the board for PR-summary metadata.
reset_into_bootloader()
¶
Try to put the board into its UF2 bootloader via the running runtime.
Issues a runtime-specific reset command
(machine.bootloader() on MicroPython,
microcontroller.on_next_reset(RunMode.BOOTLOADER) +
microcontroller.reset() on CircuitPython) and swallows
the connection-drop that follows. The serial link is torn
down as the board resets, so a clean response is not
expected.
Returns:
| Type | Description |
|---|---|
bool
|
|
bool
|
should be rebooting into its bootloader). |
bool
|
|
bool
|
bootloader-entry API, the transport could not be opened, |
bool
|
or the command failed before reaching the board. |
deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, **transport_kwargs)
¶
Write files onto the device and execute the entrypoint.
Distinct from :meth:stage + :meth:execute. The former pair
takes dirs + test files + harness source; this method takes a
generic path-to-bytes map and a single entrypoint path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
files
|
dict[str, bytes]
|
On-device-path -> file-bytes mapping. Paths may
start with |
required |
entrypoint
|
str
|
On-device path (must be a key of files) for the runtime to execute after staging. |
required |
on_file_staged
|
Callable[[str], None] | None
|
Optional per-file callback invoked with the on-device path as each file is written. |
None
|
on_execute_line
|
Callable[[str], None] | None
|
Optional callback invoked once per line of captured execute output, in order. Not guaranteed to stream live, since transports may call it after execute() completes. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
Combined stdout from the entrypoint execution. |
list_files_in_scope(*, clean_slate=False)
¶
Enumerate device files within the deploy's managed scope.
Returns the file set on the device today that the next deploy would replace. Files in this set that aren't in the next deploy's payload are the stale set to delete before writing.
clean_slate=True (the deploy default) widens the scope to
the whole device minus the closed keep set
(:data:flash_drive.DEVICE_KEEP_SET) and device-managed noise,
so a stale board settings.toml or leftover user file is
reconciled away. clean_slate=False is the additive scope
(the --no-wipe opt-out): only the entrypoint / state files
and the library tree, leaving other root files in place.
Returns paths in the same leading-slash form
:meth:deploy_files accepts ("/lib/foo.py",
"/code.py", "/active.py", etc.). Order is
unspecified; callers sort if they need deterministic output.
Transports that don't support persistent state (RAM-mode deploys: nothing survives across deploys to be diffed) return an empty list.
delete_files(paths)
¶
Delete paths from the device's filesystem.
No-op when paths is empty. Each path must be in the same
leading-slash form :meth:deploy_files accepts; transports
normalize internally. Missing paths are tolerated silently:
a previous deploy may have already removed something, so
re-deleting shouldn't error.
Best-effort: a single delete failure logs a warning but does not abort the batch. The deploy that follows still writes the new payload, and leaving a stale file in scope is preferable to skipping the deploy outright.
clear_entrypoints()
¶
Delete any persisted code.py / main.py and confirm
the deletion landed on the device before returning.
Call once before a soft-reset when a stale entrypoint from a prior deploy would otherwise run on the reboot. Filesystem transports unlink the files and verify they are gone, flushing FAT first on CIRCUITPY so the bytes hit the medium before the reboot re-reads it. RAM and mount transports never persist an entrypoint, so this is a no-op there.
wipe_filesystem()
¶
Erase the device's user filesystem before the next deploy.
Destructive: wipes every file the runtime can see, both
in-scope (/lib/*, /code.py / /main.py / etc.) and
out-of-scope (/settings.toml, hand-edited boot.py,
user-uploaded assets). Use for clean-slate / corruption-
recovery flows where an ordinary diff-deploy isn't enough,
and to reclaim flash filled with stage residue that has hit
ENOSPC mid-deploy.
Per-runtime recipe matrix:
- CircuitPython (any board):
import storage; storage.erase_filesystem()over the raw REPL. Reformats the FAT volume and triggers a hard reset; the transport swallows the USB-CDC drop and re-establishes raw REPL after the volume re-mounts. - MicroPython on rp2 (Pi Pico W et al.):
os.umount('/'); os.VfsLfs2.mkfs(rp2.Flash()); machine.soft_reset(). - MicroPython on esp32 (Lolin S2 family et al.):
os.umount('/'); os.VfsLfs2.mkfs(esp32.Partition.find( TYPE_DATA, label='vfs')[0]); machine.soft_reset(). - Other MicroPython substrates: raise
RuntimeErroruntil a verified recipe is added.
mkfs is required for MicroPython rather than a recursive
os.remove walk: LittleFS metadata + wear-leveling
artifacts survive a file-by-file delete, so a board with a
small partition (rp2's ~850 KB, esp32's 2 MB) can still hit
ENOSPC mid-deploy after a non-mkfs "wipe." Firmware
partitions are untouched on every runtime.
RAM-mode / mount-mode deploys (CP RAM, MP mount) are no-ops, since neither writes to flash, so there's nothing persistent to wipe. No need to gate on mode at the call site.
ExtendedTransportProtocol
¶
Bases: TransportProtocol, Protocol
Transport contract plus the CircuitPython RAM-mode chunking helpers.
Implemented by :class:CircuitpythonTransport. Runtime-check
against this protocol before calling the chunked-execute helpers.
execute_scripts(bootstrap_scripts, *, on_line=None)
¶
Run multiple bootstrap scripts in one interpreter session.
on_line threads through each per-script
:meth:TransportProtocol.execute call, so a chunked CP RAM-mode
deploy dispatches stdout lines across all scripts in arrival
order under a single callback.
probe_free_memory()
¶
Return free heap bytes reported by the connected board.
inline_script_budget_bytes()
¶
Return a conservative per-script budget based on live free heap.
is_in_deploy_scope(device_path)
¶
Return True when device_path falls inside the deploy's managed scope.
Scope rule:
- The four entrypoint / state files in :data:
DEPLOY_SCOPE_FILES(/code.py,/main.py,/active.py,/runtime_config.msgpack). - Everything under
/lib/; see :data:DEPLOY_SCOPE_PREFIXES.
Anything else (user-uploaded images, manually-edited
boot.py overrides, hand-tuned settings.toml knobs) is
out of scope and survives every diff-deploy untouched.
The check is path-shape only: callers should normalize their paths to leading-slash form before calling.
validate_entrypoint_in_files(files, entrypoint, *, error_cls=ValueError)
¶
Raise error_cls if entrypoint is not a key of files.
The exact message text ("entrypoint <name> missing from
files ...") is pattern-matched by
:func:~chumicro_deploy.recovery.classify_deploy_failure to route
to :attr:DeployFailureKind.CONFIGURATION_ERROR, so the message
shape is part of the helper's contract.
parse_probe_output(output)
¶
Extract a :class:DeviceImplementation from probe stdout.
Scans for the __CHU_IMPL__: marker line emitted by
:data:PROBE_IMPLEMENTATION_SCRIPT and ignores any surrounding
output. When an accompanying __CHU_UID__: line is present,
its hex payload is attached to :attr:DeviceImplementation.uid;
output without that line parses cleanly with uid="".
Returns None when the __CHU_IMPL__: marker is missing or
its payload is malformed (the "probe unavailable" signal).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
output
|
str
|
Combined stdout (and stderr if merged) from the probe
script's |
required |
write_files_to_staging(staging_path, files, on_file_staged=None)
¶
Write a device-path → bytes mapping into a host staging tree.
The shared front half of both transports' deploy_files:
sorted for deterministic staging order, leading slashes stripped
so the device-path → staging-path translation is reversible,
parent dirs created on demand, per-file callback fired after each
write.
MicroPython transport¶
chumicro_deploy.micropython_transport
¶
MicroPython device transport using mpremote.
Two execution paths:
- Persistent serial transport (mount mode and per-execute path):
uses
mpremote.transport_serial.SerialTransportdirectly. Opens the serial port once per session, enters raw REPL once, mounts the staging directory once, and runs each bootstrap viaexec_raw. Avoids the cold-start cost of spawningmpremoteperexecute()call. - Subprocess fallback (copy mode staging, reset/recover): uses the
mpremoteCLI for operations that are one-shot and easier to express via the CLI. The serial port is closed before these calls and reopened on the nextexecute().
MicropythonTransportError
¶
Bases: DeviceTransportError
Raised when an mpremote command fails.
MicropythonMidDeployDisconnected
¶
Bases: MicropythonTransportError, MidDeployDisconnected
Raised when the device drops mid-deploy.
The MicroPython face of :class:.protocol.MidDeployDisconnected
(which owns the message shape and the :attr:cause attribute).
MicropythonTransport
¶
Transport for MicroPython boards.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
address
|
str
|
Serial port or network address of the device. |
required |
baudrate
|
int
|
Serial baud rate (default 115200). Only used for the
persistent serial transport, because subprocess |
115200
|
mode
|
str
|
|
'mount'
|
runner
|
Callable[..., CompletedProcess] | None
|
Callable that executes subprocess commands. Accepts
the same signature as |
None
|
transport_factory
|
Callable[[str, int], Any] | None
|
Callable that builds a :class: |
None
|
time
|
TimeSource | None
|
Object providing |
None
|
connect()
¶
Verify the device is reachable by running a no-op command.
Uses subprocess so the persistent serial transport is opened
lazily on the first execute() (or eagerly during mount-mode
stage()). Avoids holding the serial port during the gap
between connect() and stage().
stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)
¶
Prepare a staging directory with library sources, tests, and harness.
In mount mode, the staging directory is mounted on the device
via the persistent serial transport. In copy mode, it is
recursively copied to flash via mpremote fs cp -r.
Safe to call repeatedly on the same transport (RAM-mode
orchestration re-stages per file). On re-stage in mount mode
the existing mount is dropped first. Otherwise mpremote's
mount_local hits OSError: [Errno 1] EPERM because the
device-side mount hook refuses to replace a live mount.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_dirs
|
list[Path]
|
Library |
required |
test_files
|
list[Path]
|
Test files to stage. |
required |
harness_source
|
Path
|
Path to the test harness |
required |
extra_modules
|
list[Path] | None
|
Optional sibling Python files to copy to the staging root next to the test files. |
None
|
extra_files
|
dict[str, bytes] | None
|
Non-Python files to land at named device paths
(typically |
None
|
Raises:
| Type | Description |
|---|---|
UnsupportedExtraFilesError
|
|
execute(bootstrap_script, *, on_line=None)
¶
Execute a bootstrap script on the device and return captured output.
Uses the persistent serial transport's exec_raw so each call
amortizes the one-time mpremote-cold-start cost. In copy mode
the serial transport is opened lazily (since stage() released
it for the fs cp subprocess).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
bootstrap_script
|
str
|
Python code to execute on the device. |
required |
on_line
|
Callable[[str], None] | None
|
Optional callback invoked with each stdout line as
it arrives over the serial link, before this method
returns. Lines are delivered without their trailing
newline; |
None
|
Returns:
| Type | Description |
|---|---|
str
|
Captured stdout from the device. |
run_script(script, *, timeout=10.0)
¶
Run script via the persistent serial REPL with no staging.
Opens the raw REPL if needed and runs the script directly via
exec_raw. Bypasses the stage()-first precondition
that :meth:execute enforces because script is expected to
be self-contained (no imports of staged modules).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
script
|
str
|
Self-contained Python source to run. |
required |
timeout
|
float
|
Idle-timeout for the script's output, in seconds. |
10.0
|
Returns:
| Type | Description |
|---|---|
str
|
Captured stdout from the device. |
soft_reset()
¶
Soft-reset the device to clear interpreter state.
Sends Ctrl-D through the persistent raw REPL. This clears
sys.modules and the device heap without toggling USB, so
it is safe to run between every file.
The mount is not restored by this method. Callers always
call stage() next, and stage() owns the fresh mount.
Restoring the mount here would double-wrap mpremote's
SerialIntercept (mount_local unconditionally wraps
self.serial) and garble subsequent I/O.
Leaves the interpreter clean and the serial un-wrapped. If no
persistent transport is open, falls back to a subprocess
mpremote reset with a bounded settle-and-retry: a prior hard failure
(e.g. a stage ENOSPC) can wedge the board so its USB-CDC
re-enumerates, and the first mpremote connect ... reset
then fails it may be in use by another program. Retrying
across that window keeps one library's failure from cascading
through every subsequent per-library reset.
recover()
¶
Attempt to recover after a failed test.
Closes the persistent transport (if open) and reconnects from
scratch. More aggressive than :meth:soft_reset because the
previous failure might have left the raw REPL in an unknown
state where exit_raw_repl itself could hang.
reset_into_bootloader()
¶
Issue machine.bootloader() to drop into the UF2 bootloader.
Whether the call actually puts the chip in bootloader mode
depends on the board's MicroPython build. RP2040 / RP2350
ports wire machine.bootloader() to the native UF2 ROM
bootloader and it just works. Other ports may not implement
the function at all, or may implement it as a plain reset
that boots straight back into the app. We try blindly
rather than maintain a board / firmware-version table: a
successful entry surfaces as a new ROM-bootloader serial
port visible to whatever polls for one, and an unsuccessful
entry surfaces as no-new-port. Don't add board-specific
branching here: firing machine.bootloader() blindly and
letting the caller's drive-poll be the success signal handles
every port the same way.
probe_implementation()
¶
Query sys.implementation on the board for PR-summary metadata.
Opens the persistent raw REPL if needed and runs a short inline
script. No staging required because sys.implementation is a
built-in attribute. Failures are swallowed (None returned)
so a flaky or unusual firmware never blocks the real test run.
Returns:
| Type | Description |
|---|---|
DeviceImplementation | None
|
class: |
DeviceImplementation | None
|
probe could not complete or its output did not contain the |
DeviceImplementation | None
|
expected marker line. |
disconnect()
¶
Clean up staging directory and close the persistent serial transport.
deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, follow='exec', clean=False)
¶
Write files onto the device and execute entrypoint.
In mount mode, the file tree is staged host-side, mounted
through the persistent raw REPL, and the entrypoint runs from
the mount point. In copy mode, the staging tree is pushed
to flash via mpremote fs cp -r and the entrypoint runs in
whichever follow mode follow selects.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
files
|
dict[str, bytes]
|
On-device-path -> bytes mapping. Leading slashes are stripped before joining with the staging dir or device root. |
required |
entrypoint
|
str
|
On-device path, must be a key of files. |
required |
on_file_staged
|
Callable[[str], None] | None
|
Per-file callback invoked with the on-device path as each file is written to staging. |
None
|
on_execute_line
|
Callable[[str], None] | None
|
Callback invoked once per output line, in order, after execution completes. |
None
|
follow
|
Literal['exec', 'soft_reboot']
|
Selects how the entrypoint runs.
|
'exec'
|
clean
|
bool
|
Copy mode only. When |
False
|
Returns:
| Type | Description |
|---|---|
str
|
Combined stdout captured from the entrypoint execution. |
str
|
For |
str
|
this is whatever accumulated up to self.timeout. |
Raises:
| Type | Description |
|---|---|
MicropythonTransportError
|
|
list_files_in_scope(*, clean_slate=False)
¶
Implements the contract on :meth:TransportProtocol.list_files_in_scope.
Copy mode dispatches the on-device walk via raw REPL
(:data:._device_scripts.LIST_ALL_SCRIPT for
clean_slate=True, :data:._device_scripts.LIST_SCOPE_SCRIPT
otherwise). Mount mode (RAM) returns an empty list.
Raises :class:MicropythonTransportError on raw-REPL failure.
delete_files(paths)
¶
Delete paths and reap directories that become empty.
No-op in mount (RAM) mode, because mount mode doesn't write to
flash, so there's nothing to delete. Otherwise sends a
small script that calls os.remove on each path then walks
the whole filesystem bottom-up and os.rmdir-reaps any
directory that's now empty; missing paths and per-path errors
are tolerated silently so the diff-cleanup pass never blocks
the actual deploy.
The reap is load-bearing on MicroPython: os.remove drops
files but never their parent directory, and MP's default
sys.path is ['', '.frozen', '/lib'] with '' (root)
FIRST. An empty root-level /<pkg>/ resolves
import <pkg> to a PEP 420 namespace package (__file__
is None, dir() empty) and shadows the populated
/lib/<pkg>/ deeper in the path. rmdir only removes
an empty directory so live packages are never touched.
Dot-prefixed entries are skipped: they're not ours to reap.
clear_entrypoints()
¶
Remove main.py / code.py and confirm they are gone.
Copy mode only. Mount (RAM) mode never writes a persistent
entrypoint. Runs os.remove on the device (authoritative
immediately, since unlike the CIRCUITPY host-FAT path there is
no flush lag) and, in the same raw-REPL script, re-stats each
path so a still-present entrypoint raises on the device and
surfaces as a loud error. Call once before a soft-reset so
the reboot cannot race a stale entrypoint left by a prior
deploy.
wipe_filesystem()
¶
Implements the contract on :meth:TransportProtocol.wipe_filesystem.
Dispatches :data:_WIPE_FILESYSTEM_SCRIPT as a one-shot
mpremote exec subprocess rather than over the persistent
:class:SerialTransport: machine.soft_reset() clears
interpreter state and the raw_repl framing, so reusing the
persistent transport across the reset would leave it in an
inconsistent state. After the subprocess returns, the serial
port is given :data:_WIPE_REBOOT_SETTLE_SECONDS to settle so
the next :meth:connect / :meth:stage call sees a fully
booted runtime ready to accept a fresh raw-REPL session.
CircuitPython transport¶
chumicro_deploy.circuitpython_transport
¶
CircuitPython device transport using pyserial raw REPL.
Uses pyserial to connect to CircuitPython boards via serial and
execute code through raw REPL mode (Ctrl-A). Two modes:
- RAM mode (default): All source code is sent inline through the raw REPL, with no file copy or flash writes required.
- Flash mode: Files are copied to the CIRCUITPY USB drive for persistent deployment. Autoreload is managed via raw REPL commands.
Raw REPL protocol:
1. Ctrl-C × 2 interrupts any running code.
2. Ctrl-A enters raw REPL (prompt: raw REPL; CTRL-B to exit\r\n>).
3. Send code bytes, terminated with Ctrl-D.
4. Response: OK<stdout>\x04<stderr>\x04>.
PostStageStep
¶
Bases: Enum
Which of the two post-rsync steps a deploy context runs: soft-
reboot into the freshly-staged code.py, or keep the live raw-
REPL session for the harness to drive.
Every context stages through the same clean-slate rsync +
:data:flash_drive.DEVICE_KEEP_SET, so the bytes reach the board
identically for a project, an example, and a functional test.
Only the step taken once the bytes have landed differs, and only
two values exist. Naming them here keeps the divergence explicit
and gives the planned drift lint a symbol to anchor on instead of
two unrelated methods that happen to differ.
- :attr:
SOFT_REBOOT_AND_TAIL: Ctrl-D soft-reboot so the board runs the freshly-stagedcode.py, then capture its serial output. - :attr:
HARNESS_EXEC_OVER_REPL: keep the live raw-REPL session alive (a soft-reboot would tear down state the harness drives over that session), refresh the FAT cache with a cheap directory walk, and let the caller exec the harness and collect asserts.
The chosen value never changes how the bytes got there: everything up to and including the rsync runs the same code path.
SerialPort
¶
Bases: Protocol
Structural interface for a serial port.
Implementations can satisfy this protocol without importing pyserial.
CircuitpythonTransportError
¶
Bases: DeviceTransportError
Raised when a CircuitPython serial operation fails.
CircuitpythonMidDeployDisconnected
¶
Bases: CircuitpythonTransportError, MidDeployDisconnected
Raised when the device drops mid-deploy.
The CircuitPython face of :class:.protocol.MidDeployDisconnected
(which owns the message shape and the :attr:cause attribute).
The wrapped :class:OSError is typically a
:class:serial.SerialException here.
CircuitpythonTransport
¶
Transport for CircuitPython boards via pyserial raw REPL.
Supports two modes:
- ram (default): all source code is sent inline through the raw REPL, with no file copy or mounting needed.
- flash: files are copied to the CIRCUITPY USB drive for persistent deployment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
address
|
str
|
Serial port path (e.g. |
required |
baudrate
|
int
|
Serial baud rate. Defaults to 115200. |
115200
|
timeout
|
float
|
Read timeout in seconds. |
DEFAULT_TIMEOUT
|
mode
|
str
|
|
'ram'
|
serial_port_factory
|
Callable[..., object] | None
|
Callable that creates a serial port object.
Accepts |
None
|
time
|
TimeSource | None
|
Object providing |
None
|
The CIRCUITPY drive (mode="flash") is auto-resolved at deploy
time via :func:_circuitpy_volume_candidates and verified against
the connected board's identity by :meth:_verify_drive_for_board.
No per-transport drive path is stored. Resolving by board UID at
deploy time lets a host with two boards pick the right drive even
when the OS gives the same mount name to a different board across
reboots.
staged_sources
property
¶
Return the staged module sources, or None if not staged.
connect()
¶
Open the serial port and enter raw REPL mode.
Sends Ctrl-C × 2 to interrupt any running code, then Ctrl-A to enter raw REPL.
Raises:
| Type | Description |
|---|---|
CircuitpythonTransportError
|
If the serial port cannot be opened or raw REPL prompt is not received. |
stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)
¶
Read source files into memory for inline execution.
Sister of :meth:deploy_files that takes library/test/harness
directories (source_dirs + test_files + harness_source)
instead of a flat files: dict[device_path, bytes].
In RAM mode, source code is read and stored for embedding into the bootstrap code block sent via raw REPL.
In flash mode, source packages are copied to the CIRCUITPY USB drive after disabling autoreload.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_dirs
|
list[Path]
|
Library |
required |
test_files
|
list[Path]
|
Test files to stage (stored for bootstrap generation). |
required |
harness_source
|
Path
|
Path to the test harness |
required |
extra_modules
|
list[Path] | None
|
Optional sibling Python files to register as
importable on the device alongside library sources.
In RAM mode they join |
None
|
extra_files
|
dict[str, bytes] | None
|
Non-Python files to land at named device paths,
typically |
None
|
Raises:
| Type | Description |
|---|---|
UnsupportedExtraFilesError
|
|
execute(bootstrap_script, *, on_line=None)
¶
Send a code block through raw REPL and return captured stdout.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
bootstrap_script
|
str
|
Python code to execute on the device. |
required |
on_line
|
Callable[[str], None] | None
|
Optional callback invoked with each stdout line as
it arrives over the serial link, before this method
returns. Lines are delivered without their trailing
newline; |
None
|
Returns:
| Type | Description |
|---|---|
str
|
Captured stdout from the device. |
Raises:
| Type | Description |
|---|---|
CircuitpythonTransportError
|
If stage() has not been called, the device returns an error, or communication fails. |
execute_scripts(bootstrap_scripts, *, on_line=None)
¶
Execute multiple raw-REPL scripts in one interpreter session.
Large CircuitPython RAM-mode payloads are more reliable when split into smaller scripts. Each script leaves the interpreter in raw REPL ready for the next chunk.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
bootstrap_scripts
|
list[str]
|
Ordered raw-REPL scripts to execute. |
required |
on_line
|
Callable[[str], None] | None
|
Optional callback threaded through each per-script
:meth: |
None
|
Returns:
| Type | Description |
|---|---|
str
|
Stdout from the final script. |
reset_into_bootloader()
¶
Reset into the UF2 bootloader via microcontroller module.
microcontroller.on_next_reset(RunMode.BOOTLOADER) +
microcontroller.reset() is CircuitPython's documented
way to enter bootloader mode. CP implements it across all
ESP32-S2/S3/C6/P4 + RP2040/RP2350 + most SAMD/nRF52 ports,
but whether the call actually puts a particular board in
bootloader mode depends on that board's HAL build. Treat
success as observable, not declared: the raw-REPL link drops
as the chip resets (expected), and a new bootloader port /
UF2 drive becomes visible to whatever polls for one. Don't
add board-specific branching here.
Closes the serial port directly so a subsequent
:meth:disconnect becomes a no-op. The USB link is gone on
purpose and running the normal restore dance (_enter_raw_repl
+ autoreload-on + Ctrl-D) against a dying link only produces
misleading warnings.
probe_implementation()
¶
Query sys.implementation on the board for PR-summary metadata.
Uses the persistent raw REPL _send_repl_command helper so
no staging is required. sys.implementation is a built-in.
Failures are swallowed (None returned) so a flaky firmware
never blocks the real test run.
Returns:
| Type | Description |
|---|---|
DeviceImplementation | None
|
class: |
DeviceImplementation | None
|
the probe could not complete. |
probe_free_memory()
¶
Return free heap bytes reported by the connected board.
inline_script_budget_bytes()
¶
Return a conservative raw-REPL script budget based on live heap.
run_script(script, *, timeout=10.0)
¶
Run script via the persistent raw REPL with no staging.
Bypasses the stage()-first precondition that
:meth:execute enforces. script must be self-contained
(no imports of staged modules).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
script
|
str
|
Self-contained Python source to run. |
required |
timeout
|
float
|
Reserved for protocol parity; the underlying
|
10.0
|
Returns:
| Type | Description |
|---|---|
str
|
Captured stdout from the device. |
soft_reset()
¶
Soft-reset the interpreter and re-enter raw REPL.
Flushes any host-cached FAT writes to the CIRCUITPY volume
(flash mode only), exits raw REPL (Ctrl-B), sends Ctrl-D to
trigger a soft reboot (which clears sys.modules and all
interpreter state), waits for the reboot to complete, then
re-enters raw REPL.
Leaves the interpreter clean, with previous modules evicted from RAM.
Raises:
| Type | Description |
|---|---|
CircuitpythonTransportError
|
If raw REPL cannot be re-established after the reset. |
recover()
¶
Attempt to recover raw REPL after a failed test.
Sends Ctrl-C to interrupt any running code, drains stale output, then re-enters raw REPL mode. Call this after a test error before running the next test.
Raises:
| Type | Description |
|---|---|
CircuitpythonTransportError
|
If raw REPL cannot be re-established. |
deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, tail_seconds=None, clean=False)
¶
Deploy files and execute entrypoint in the configured mode.
In flash mode, every entry of files is written to the CIRCUITPY USB drive (auto-detecting the mount when not configured), the volume is flushed, and the entrypoint runs through the persistent raw REPL. Autoreload is disabled during writes so the board does not reset mid-deploy.
In RAM mode, no filesystem is touched. Every
non-entrypoint .py file is injected into sys.modules
via the class-as-module pattern (see
:func:build_circuitpython_deploy_scripts) and the entrypoint
runs as __main__. Non-.py payload is silently skipped.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
files
|
dict[str, bytes]
|
On-device-path -> bytes mapping. In flash mode the
leading slash is stripped before joining with the
drive mount. In RAM mode the path derives the dotted
module name ( |
required |
entrypoint
|
str
|
On-device path, must be a key of files. |
required |
on_file_staged
|
Callable[[str], None] | None
|
Per-file callback after each write (flash mode) or before the inline scripts run (RAM mode, in sorted-key order). |
None
|
on_execute_line
|
Callable[[str], None] | None
|
Callback invoked once per captured output line, in order, after the entrypoint returns. |
None
|
tail_seconds
|
float | None
|
Flash mode only. Override for how long
serial output is captured after the soft-reboot.
|
None
|
clean
|
bool
|
Flash mode only. When |
False
|
Returns:
| Type | Description |
|---|---|
str
|
Combined stdout from the entrypoint execution. |
Raises:
| Type | Description |
|---|---|
CircuitpythonTransportError
|
Port not connected, entrypoint missing from files, or (flash mode) the CIRCUITPY drive cannot be located. |
list_files_in_scope(*, clean_slate=False)
¶
Implements the contract on :meth:TransportProtocol.list_files_in_scope.
Flash mode walks the CIRCUITPY USB drive directly via stdlib
pathlib, which is faster and simpler than a raw-REPL
round-trip, and the drive's contents are the device's
filesystem. Pairs :meth:_resolve_circuitpy_drive with
:meth:_verify_drive_for_board so the macOS multi-board
mount-order swap is silently corrected to the connected
board's actual mount before the walk. RAM mode returns an
empty list.
delete_files(paths)
¶
Delete paths from the CIRCUITPY drive.
Flash mode only. RAM mode is a no-op since nothing was ever written to flash. Each path is normalized to a leading-slash form, joined under the CIRCUITPY mount point, and unlinked best-effort. Missing paths and per-path errors are tolerated silently so a transient I/O hiccup never blocks the deploy that follows.
Pairs _resolve_circuitpy_drive with _verify_drive_for_board
so the macOS first-mount-wins behavior cannot unlink the
other board's files on multi-board hosts.
Uses :meth:pathlib.Path.unlink rather than rsync --delete
on purpose: rsync's delete semantics are "remove anything in
DEST not in SRC", which is the wrong shape for "delete these
specific files." Unlink also dodges FAT32's data-write
reliability concerns by only touching directory entries, no
payload bytes. The stale set is recomputed on every deploy,
so a swallowed error here just retries next time.
clear_entrypoints()
¶
Unlink code.py / main.py and confirm they are gone.
Flash mode only. RAM mode never writes a persistent
entrypoint, so there is nothing to clear. Host-side
pathlib.unlink (a directory-entry-only op with no FAT data
write), then flush_volume so the deletion is on the
physical medium before a soft-reboot re-reads FAT, then a
bounded poll that the drive no longer shows them. Raises if
an entrypoint is still present after the flush + poll budget,
because a surviving code.py is exactly the soft-reboot race
this removes.
wipe_filesystem()
¶
Implements the contract on :meth:TransportProtocol.wipe_filesystem.
Drives the on-board nuclear option through raw REPL: the call
reformats the FAT volume and reboots the board. The host-side
serial session goes away mid-call as USB-CDC drops, and the
failure that surfaces is expected and swallowed. After
waiting for the reformat + reboot to settle the port is
re-opened and raw REPL re-entered, leaving the transport in
the same state :meth:connect does so a follow-up
:meth:deploy_files works without further setup.
disconnect()
¶
Close the serial port and clear staged data.
Pure teardown. Sends a bare Ctrl-B (exits raw REPL when in
raw REPL, a no-op control byte otherwise) so the next serial
consumer never finds the board parked in raw REPL, then
closes the port. No Ctrl-C, because that would interrupt any
code.py left running on the board. No
supervisor.runtime.autoreload = True, because flipping
autoreload back on outside an active raw-REPL session can layer
an autoreload-driven soft-reboot on top of one already in
flight, which can wedge ESP32-S2 USB-CDC firmware.
Tolerates an already-closed port (e.g. after
:meth:reset_into_bootloader nulled :attr:_port): the
method finds nothing to close and only clears
:attr:_staged_sources.
CircuitPython bootstrap helpers¶
chumicro_deploy.circuitpython_bootstrap
¶
CircuitPython bootstrap code generator.
Generates inline code blocks that can be sent through the CircuitPython raw REPL. Two builders share the same helper-function prefix:
- :func:
build_circuitpython_bootstrap_scriptsis the test-harness flavor. Registers library modules via the class-as-module pattern (notypes.ModuleTypeon CircuitPython), inlines a test file, and runs it throughchumicro_test_harness.runner.run_module. - :func:
build_circuitpython_deploy_scriptsis the deploy flavor. Registers every non-entrypoint file as an importable module, thenexec()-s the entrypoint as__main__. No test-harness dependency is needed, and the deploy tail is two lines long, kept as an inline string constant rather than a separate template file.
Both share the static helper functions in
circuitpython_bootstrap_template.txt (read once per builder
call). _register_stub and _populate_module are universal.
Only the dynamic parts (module sources, entrypoint / test source,
filter) are generated here.
CircuitpythonBootstrapTooLargeError
¶
Bases: ValueError
Raised when an inline CircuitPython bootstrap is too large to send safely.
build_circuitpython_bootstrap_scripts(staged_sources, test_file, *, name_filter=None, max_chunk_size_bytes=DEFAULT_INLINE_SCRIPT_BUDGET_BYTES)
¶
Generate chunked raw-REPL scripts for CircuitPython RAM-mode tests.
CircuitPython raw REPL buffers and compiles each submitted script before it executes. Large one-shot payloads can therefore exhaust heap even when the final imported modules would fit. This builder splits the inline bootstrap into smaller scripts that can be executed sequentially in one interpreter session.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
staged_sources
|
list[tuple[str, str]]
|
List of |
required |
test_file
|
Path
|
Path to the test file to execute. |
required |
name_filter
|
str | None
|
Optional substring filter passed to |
None
|
max_chunk_size_bytes
|
int
|
Maximum encoded size for any single submitted raw REPL script. |
DEFAULT_INLINE_SCRIPT_BUDGET_BYTES
|
Returns:
| Type | Description |
|---|---|
list[str]
|
Ordered list of Python source strings to execute sequentially. |
build_circuitpython_deploy_scripts(files, entrypoint, *, max_chunk_size_bytes=DEFAULT_INLINE_SCRIPT_BUDGET_BYTES)
¶
Generate raw-REPL scripts for CircuitPython RAM-mode deploy.
The returned scripts inject every non-entrypoint file as an
importable module via sys.modules, then exec() the
entrypoint as __main__. No chumicro_test_harness
dependency is required on the device.
Module naming follows CircuitPython's sys.path convention:
files under /lib/ are stripped of that prefix, so
/lib/foo.py registers as foo and
/lib/foo/bar.py as foo.bar. Files at the device root
(/foo.py) register as top-level foo. Non-.py files
are silently skipped. RAM-mode deploy has no device filesystem
to write them to, so shipping non-Python assets requires flash
mode.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
files
|
dict[str, bytes]
|
On-device-path -> bytes mapping. |
required |
entrypoint
|
str
|
On-device path exec'd as |
required |
max_chunk_size_bytes
|
int
|
Maximum encoded size for any single
submitted raw REPL script. Defaults to
:data: |
DEFAULT_INLINE_SCRIPT_BUDGET_BYTES
|
Returns:
| Type | Description |
|---|---|
list[str]
|
Ordered list of Python source strings to execute sequentially. |
Raises:
| Type | Description |
|---|---|
ValueError
|
entrypoint is missing from files or max_chunk_size_bytes is not positive. |
CircuitpythonBootstrapTooLargeError
|
A single chunk exceeds max_chunk_size_bytes. |
Flash drive helpers¶
chumicro_deploy.flash_drive
¶
CircuitPython flash-mode USB-drive staging and FAT32 hygiene helpers.
Every helper here operates on filesystem paths or subprocess calls
and has no transport state to carry, so they live as module-level
functions instead of methods on a class. :func:flush_volume takes
an injected sleep callable so tests can skip the real settle delay.
FlashDriveError
¶
Bases: Exception
Raised when a flash-drive staging operation fails.
FatVolumeCorruptionError
¶
Bases: FlashDriveError
Raised when a FAT volume carries torn directory entries.
A torn entry is one the host can list via readdir but whose
stat fails with EINVAL and which cannot be unlinked, the
state a FAT directory is left in when cached host-side metadata
writes land across a board reset. The volume cannot be repaired
in place; recovery is a full board-filesystem reformat.
Attributes:
| Name | Type | Description |
|---|---|---|
corrupted_paths |
Volume-relative paths of the torn entries,
as enumerated by :func: |
compute_rsync_timeout_seconds(staging_size_bytes)
¶
Pick an rsync timeout proportional to staging-tree size.
Formula::
max(RSYNC_TIMEOUT_BASE_SECONDS + size_mb * RSYNC_TIMEOUT_PER_MB_SECONDS,
RSYNC_TIMEOUT_MIN_SECONDS)
Scales the deadline so fast boards fail fast on a real wedge while slow boards still have headroom to finish.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
staging_size_bytes
|
int
|
Sum of file sizes in the local staging
tree, typically from :func: |
required |
merge_packages(source_directory, staging_destination, *, target_runtime=None, include_test_support=False)
¶
Copy top-level packages from a source directory to a staging dir.
Merges into the staging destination using dirs_exist_ok=True so
multiple source directories can contribute packages. Operates on
the local filesystem (not the USB drive), so shutil.copytree
is reliable here.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_directory
|
Path
|
A |
required |
staging_destination
|
Path
|
Local staging directory to merge into. |
required |
target_runtime
|
str | None
|
When set, |
None
|
include_test_support
|
bool
|
When |
False
|
rsync(source, destination, *, delete=True, additional_excludes=(), timeout=None)
¶
Rsync a source directory's contents to a destination.
Single FAT-write primitive: --checksum verifies content
(FAT32 timestamps are unreliable) and --inplace avoids
temp-file rename races on FAT32. Two parameter shapes:
- Clean push:
delete=Truewithadditional_excludes=DEVICE_KEEP_SET. Clean slate, only the closed keep set survives. - Additive push:
delete=Falsewith no extra excludes. Stale files persist. Used when other board files are hand-managed.
The base exclude set (build artifacts + macOS noise / sentinel
dirs) is shared and unconditional; see :data:_BASE_RSYNC_EXCLUDES.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
Path
|
Source directory whose contents to sync. |
required |
destination
|
Path
|
Destination directory. |
required |
delete
|
bool
|
When |
True
|
additional_excludes
|
tuple[str, ...] | list[str]
|
Extra basenames to add to |
()
|
timeout
|
float | None
|
Override the auto-computed timeout (seconds). Default
|
None
|
Raises:
| Type | Description |
|---|---|
FlashDriveError
|
If rsync is not installed or the sync fails. |
verify_rsync(source, destination, *, additional_excludes=(), timeout=30.0)
¶
Confirm destination's contents match source via rsync dry-run.
Runs rsync --recursive --checksum --dry-run --itemize-changes
(the same flags :func:rsync uses, plus dry-run + itemize) and
returns the list of paths rsync reports as needing update. When
the previous real :func:rsync call committed every byte, the
list is empty. Non-empty means content on the volume diverged
from the staging tree, which is the signal for FAT corruption,
USB-MSC partial-write, or a drive that quietly went read-only
after the writes started.
Itemize-changes flag positions: position 1 is the update marker
(< / > / c / h mean "would transfer"; . /
* mean "no update needed"). We filter on the first character
so cosmetic time / permission deltas (.f..T....) don't fire
a false positive. Only real content / size diffs do.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
Path
|
Staging tree that was rsynced to destination. |
required |
destination
|
Path
|
Mount point that should now mirror source. |
required |
additional_excludes
|
tuple[str, ...] | list[str]
|
Extra basenames passed through to
|
()
|
timeout
|
float
|
Subprocess deadline (seconds). Verification reads every file from the FAT volume. |
30.0
|
Returns:
| Type | Description |
|---|---|
list[str]
|
Sorted list of source-relative paths that the verification |
list[str]
|
rsync would update. Empty on success. |
Raises:
| Type | Description |
|---|---|
FlashDriveError
|
rsync is missing or the subprocess errored for a reason other than "would-update detected" (which shows up in stdout, not as a non-zero exit). |
format_fat_corruption_error(drive_path, corrupted_paths, *, detail='')
¶
Build the single actionable message for a torn-FAT volume.
Names every corrupted path plus the one recovery command that
works, so a sweep fails once with instructions instead of every
test failing with repeated rsync stderr noise. The lead phrase
FAT directory entries is load-bearing: the recovery
classifier's FAT_VOLUME_CORRUPT pattern matches on it, so
reword only together with the matching pattern in recovery.py.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
drive_path
|
Path
|
Mount point of the corrupted volume. |
required |
corrupted_paths
|
Iterable[str]
|
Volume-relative paths of the torn entries. May be empty when only rsync stderr identified the state. |
required |
detail
|
str
|
Optional raw diagnostic (typically rsync stderr) to embed for the report. |
''
|
scan_for_torn_directory_entries(drive_path, *, scandir=os.scandir)
¶
Stat every entry under drive_path and collect EINVAL casualties.
Walks the volume with scandir and calls stat on each entry.
A healthy entry stats cleanly; a torn FAT directory entry is
listed by readdir but fails stat with EINVAL, so any
EINVAL here is the corruption signature. Directories whose
own listing fails with EINVAL are recorded the same way.
Other per-entry OSError values (a file deleted mid-walk, a
kernel-locked macOS noise directory) are skipped: they are not
the torn-entry state this scan exists to find.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
drive_path
|
Path
|
Mount point to walk. |
required |
scandir
|
Callable[[Path], Iterable]
|
Directory-listing callable, |
scandir
|
Returns:
| Type | Description |
|---|---|
list[str]
|
Sorted volume-relative paths of every torn entry. Empty on a |
list[str]
|
healthy volume. |
strip_extended_attributes(path)
¶
Remove macOS extended attributes from all files under path.
Extended attributes (xattrs) cause slow transfers to FAT32 volumes
and generate ._ resource fork files. Stripping them from the
staging directory before rsync prevents these artifacts from
reaching the device.
No-op on non-macOS platforms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
Path
|
Root directory to strip recursively. |
required |
clean_dot_files(drive_path)
¶
Merge or remove ._ resource fork files on a FAT32 volume.
macOS creates ._ files on FAT32 drives even when rsync excludes
them, because the OS itself writes them during filesystem
operations. dot_clean merges these back into the native file
or removes them if the native file is absent.
No-op on non-macOS platforms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
drive_path
|
Path
|
Mount point of the FAT32 volume. |
required |
plant_macos_sentinels_in_staging(staging_path)
¶
Plant macOS skip-sentinels into a local staging directory.
The sentinels (see :data:_MACOS_SKIP_SENTINELS for what each
suppresses) go at the staging-tree root so rsync ships them onto
CIRCUITPY in the same pass as the payload. No host-side write to
the live drive before rsync starts, since every such write can
wedge rsync in uninterruptible kernel I/O.
No-op on non-macOS platforms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
staging_path
|
Path
|
Local staging-tree root. rsync will copy everything under this into the CIRCUITPY drive root. |
required |
cleanup_macos_noise_dirs_post_rsync(drive_path)
¶
Remove already-accumulated macOS noise directories from the drive.
Called after rsync so the wedge-risky on-drive writes stay
inside the single rsync pass. Clears the noise dirs a drive
picked up from earlier macOS mounts (:data:_MACOS_NOISE_DIRS);
rsync --delete can't, since they're excluded and partly
kernel-locked on FAT. shutil.rmtree(ignore_errors=True) walks
them best-effort, so a drive the sentinels already keep clean is a
no-op.
No-op on non-macOS platforms.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
drive_path
|
Path
|
Mount point of the CIRCUITPY drive. |
required |
flush_volume(drive_path, *, sleep, settle_delay=FLUSH_SETTLE_DELAY)
¶
Flush pending writes to the volume containing drive_path.
Three layers, weakest to strongest: the sync command (or
os.sync() off macOS) schedules every volume's dirty buffers;
F_FULLFSYNC on the mount point (via
:func:_force_full_flush_to_media) waits until this volume's
writes have actually reached the medium; the settle delay gives
the board's USB-MSC controller time to finish its own FAT
bookkeeping. All three must have completed before anything
resets the board, because a reset that lands while host-side FAT
metadata is still cached can tear directory entries into an
unreadable, undeletable state.
The settle delay goes through the injected sleep callable so tests can use a fake time source to skip it without sleeping for real.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
drive_path
|
Path
|
Path on the volume to flush. |
required |
sleep
|
Callable[[float], None]
|
Callable that sleeps for the given number of seconds. |
required |
settle_delay
|
float
|
Seconds to wait after the sync. |
FLUSH_SETTLE_DELAY
|
Test fakes¶
chumicro_deploy.testing
¶
Test fakes for the device transport layer.
Exports :class:FakeTransport, :class:FakeSerialPort,
:class:FakeTime, and :func:isolate_from_host_filesystem. See
each one's docstring for shape and usage. Typical injection::
from chumicro_deploy.testing import FakeSerialPort, FakeTime
transport = CircuitpythonTransport(
address="/dev/cu.fake",
port_factory=lambda *_args, **_kwargs: FakeSerialPort(...),
time=FakeTime(),
)
FakeTime
¶
Deterministic seconds-domain time source for host-side tests.
Bundles monotonic() and sleep() into a single injectable
object that satisfies :class:chumicro_deploy.protocol.TimeSource
(both device transports take one). monotonic()
is stable: repeated calls return the same value until the
clock is explicitly advanced. sleep(duration) auto-advances
the clock by duration without any real wait, so production
code that sleeps still moves the fake clock forward.
advance(seconds) moves the clock forward explicitly, for
scenarios where production does not sleep but the test needs to
simulate elapsed time (e.g., timeout expiry).
Example::
fake = FakeTime()
assert fake.monotonic() == 0.0
fake.sleep(1.5)
assert fake.monotonic() == 1.5
fake.advance(0.5)
assert fake.monotonic() == 2.0
__init__(start=0.0)
¶
Create a fake time source starting at start seconds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
start
|
float
|
Initial monotonic value in seconds. |
0.0
|
monotonic()
¶
Return the current fake time in seconds.
sleep(duration)
¶
Advance the clock by duration seconds (no wall-clock wait).
advance(seconds)
¶
Move the clock forward by seconds.
FakeSerialPort
¶
Simulates a pyserial Serial port for transport testing.
Records all writes and returns canned responses for reads.
Each entry in read_responses is either bytes (returned
verbatim on the matching read() call) or an instance of
BaseException (raised on that call). The exception form
scripts "first read returns bytes, second read drops the cable"
scenarios without subclassing the fake.
Instances are callable and return themselves, so a
FakeSerialPort instance can be passed directly as a transport's
serial_port_factory kwarg with no wrapper closure needed. Pass
open_error to script "factory raises when the transport asks for
a port" without writing a custom closure.
in_waiting
property
¶
Return how many bytes are available to read.
Scripted BaseException entries are reported as having
in_waiting == 1 so polling loops actually call
:meth:read (which then raises) instead of looping past
the scripted disconnect.
__call__(*_args, **_kwargs)
¶
Return this port, or raise the scripted open_error.
Lets the instance double as a serial_port_factory callable.
Accepts both positional and keyword args (different transports
call factories with different signatures); all are ignored.
Use an explicit closure when factory-arg assertions are needed.
read(size=1)
¶
Return the next canned response, or raise the scripted exception.
write(data)
¶
Record a write, or raise the configured exception.
close()
¶
Mark the port as closed.
reset_input_buffer()
¶
No-op for fake.
FakeTransport
dataclass
¶
In-memory fake that records transport calls and returns canned output.
Satisfies both :class:TransportProtocol and
:class:ExtendedTransportProtocol, so the same fake stands in
for either a MicroPython or CircuitPython transport. The
chunked-execute helpers fall back to :meth:execute when no
chunked-specific behavior is configured.
connect()
¶
Record a connect call; raise :attr:connect_raises if set.
stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)
¶
Record a stage call.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_dirs
|
list[Path]
|
Library source directories. |
required |
test_files
|
list[Path]
|
Test file paths. |
required |
harness_source
|
Path
|
Harness source directory. |
required |
extra_modules
|
list[Path] | None
|
Sibling Python files to register as importable on the device. |
None
|
extra_files
|
dict[str, bytes] | None
|
Non-Python files to land at named device paths
(typically |
None
|
Raises:
| Type | Description |
|---|---|
UnsupportedExtraFilesError
|
|
execute(bootstrap_script, *, on_line=None)
¶
Record an execute call and return canned output.
Returns the head of :attr:outputs when non-empty (popped),
otherwise :attr:execute_output. Raises :attr:execute_raises
after recording when set.
When on_line is provided, dispatches one call per line of the
returned string (split via :meth:str.splitlines) before
returning, the same shape the real transports use, so a test can
drive marker-style stdout coordination against the fake without
wiring a real serial stream.
run_script(script, *, timeout=10.0)
¶
Record a run_script call and return canned output.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
script
|
str
|
Self-contained Python source to run with no staging. |
required |
timeout
|
float
|
Recorded with the call but not enforced, since the fake transport's "device" responds instantly. |
10.0
|
Returns:
| Type | Description |
|---|---|
str
|
The configured |
str
|
the same canned-response slot; tests that need separate |
str
|
run_script / execute outputs can override after |
str
|
construction). |
execute_scripts(bootstrap_scripts, *, on_line=None)
¶
Record a chunked-execute call and return the configured output.
Raises :attr:execute_raises after recording when set.
When :attr:outputs is non-empty, pops a single head value and
returns it, treating the whole chunked execute as one batched
operation, with no synthetic per-script execute entries.
Otherwise records synthetic per-script execute entries (one
per chunk) and returns :attr:execute_output.
on_line mirrors the real-transport pass-through: the batched
head dispatches one call per line; the per-script branch threads
the callback into each :meth:execute call.
probe_free_memory()
¶
Record a probe call and return the configured free-heap value.
inline_script_budget_bytes()
¶
Return half the configured free-memory budget.
probe_implementation()
¶
Record a probe call and return the canned result.
soft_reset()
¶
Record a soft_reset call.
recover()
¶
Record a recover call; raise :attr:recover_raises if set.
disconnect()
¶
Record a disconnect call.
reset_into_bootloader()
¶
Record the call and return the configured result.
Defaults to True (pretending the dispatch succeeded) for
the common branch; override via :attr:bootloader_reset_result
to simulate the "runtime doesn't support bootloader entry"
fallback.
deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, follow='exec', clean=False, tail_seconds=None)
¶
Record a deploy_files call and return the configured output.
Emits on_file_staged per file (sorted for deterministic
order) and on_execute_line per line of execute_output
before returning. The calls entry uses a dict-of-bytes +
entrypoint + follow tuple so the payload, callback ordering,
and follow mode are all observable.
The follow kwarg accepts the same values as
:class:MicropythonTransport.deploy_files ("exec" or
"soft_reboot"). The fake records it but doesn't
otherwise change behavior.
tail_seconds is recorded on :attr:last_tail_seconds so
tests can assert the capture-window override.
list_files_in_scope(*, clean_slate=False)
¶
Return on-device paths in scope, drawn from :attr:device_files.
clean_slate=True widens to every device file except the
closed keep set (:data:flash_drive.DEVICE_KEEP_SET).
False keeps the :func:is_in_deploy_scope filter so the
additive path leaves out-of-scope files alone.
delete_files(paths)
¶
Remove paths from the simulated on-device state.
clear_entrypoints()
¶
Drop the simulated code.py / main.py entrypoints.
wipe_filesystem()
¶
Erase every simulated on-device file.
In flash/copy mode the whole user filesystem (in-scope + out- of-scope alike) is gone after this returns. In RAM/mount mode the call is a no-op, so both branches exercise against the same fake.
isolate_from_host_filesystem(monkeypatch, *, circuitpy_drives=None)
¶
Sever flash_drive's links to the real macOS host filesystem.
Two cleanups in one, both load-bearing for hermetic tests:
- macOS shell helpers → instant fakes.
flash_drivecallssubprocess.run(["sync"])/["xattr", ...]/["dot_clean", ...]/["mdutil", ...]on macOS. Each blocks for seconds on a busy host (concurrent test workers, IDE indexing, git activity). Realrsyncis left intact, because tests rely on its file-copy side effect. - CIRCUITPY drive scanning → controllable. Production calls
:func:
chumicro_deploy.circuitpy_drive._circuitpy_volume_candidatesto find/Volumes/CIRCUITPY*mounts. On a dev box with a real board plugged in, an unstubbed test will write+unlink probe files onto the real device's filesystem. Pass circuitpy_drives to point at a test-controlled directory; passNone(default) to make scans return empty so tests don't depend on host state.
Per-test monkeypatches override these. Calling
monkeypatch.setattr("chumicro_deploy.circuitpy_drive
._circuitpy_volume_candidates", ...) again later in the same test
replaces the stub for that test only.
Typical usage as a module-level autouse fixture::
@pytest.fixture(autouse=True)
def _isolate(monkeypatch):
isolate_from_host_filesystem(monkeypatch)