Skip to content

API Reference

Auto-generated from docstrings via mkdocstrings. All public names are re-exported at the package top level via the lazy-attr table in chumicro_deploy/__init__.py; the per-module sections below mirror the internal layout for readers who want to navigate by source file.

Device

chumicro_deploy.device

Device: transport-agnostic configuration for a target board.

A :class:Device bundles the runtime identity (circuitpython / micropython), the host-side connection details (serial address, baudrate, CIRCUITPY drive path), and the deploy-mode preference. :meth:Device.create_transport returns a concrete :class:~chumicro_deploy.protocol.TransportProtocol instance, so callers don't have to branch on runtime themselves.

Two fields, entrypoint_name and resource_prefix, aren't read by :meth:create_transport. They are deploy-config carried on Device so a deploy takes one configured object.

Device dataclass

Configuration for a target board.

Constructed explicitly in code, from a dict (Device.from_dict(...)), or via the built-in devices.yml loader (chumicro_deploy.config.default.load_devices_yml), or a third-party loader registered through the chumicro_deploy.config_loaders entry-point group.

Attributes:

Name Type Description
transport str

Runtime identifier, "circuitpython" or "micropython".

address str

Serial port path ("/dev/cu.usbmodem..." on macOS, "/dev/ttyACM0" on Linux, "COM3" on Windows).

baudrate int

Serial baudrate. Only meaningful for the CircuitPython transport; the MicroPython transport uses mpremote defaults.

deploy_mode str

"ram" or "flash". Mapped to the transport's mode label at construction time (mount/copy on MP, ram/flash on CP).

entrypoint_name str | None

Top-level script the runtime executes on boot. Defaults vary per runtime ("code.py" on CircuitPython, "main.py" on MicroPython). Pass None to use the runtime default.

resource_prefix str

On-device directory where library files land at deploy time.

transport_factory Callable[[Device], TransportProtocol] | None

Override hook for :meth:create_transport. When set, :meth:create_transport returns transport_factory(self) instead of building the runtime-specific transport. Per-instance, so multiple Devices can carry different transport implementations.

The CIRCUITPY drive (CP flash-mode deploys) is auto-resolved at deploy time by scanning the host's mount points and matching the connected board's UID / machine string against each candidate's boot_out.txt, so no per-device drive path is configured here.

effective_entrypoint property

Return the entrypoint filename, resolving runtime default when unset.

CircuitPython boards boot code.py; MicroPython boards boot main.py. Override via :attr:entrypoint_name when the target runs something else.

from_dict(data) classmethod

Construct a :class:Device from a mapping of field names.

Accepts the same keys as the constructor: transport, address, and optional baudrate, deploy_mode, entrypoint_name, and resource_prefix. Unknown keys are ignored so YAML / TOML / JSON inputs with extra metadata fields (id, description, circuitpy_drive_path, etc.) pass through without filtering.

Parameters:

Name Type Description Default
data Mapping[str, Any]

Mapping of field names to values. transport and address are required; everything else falls back to the constructor default.

required

Raises:

Type Description
ValueError

Missing transport or address, or any of the constructor's normal validation (unsupported transport / deploy_mode).

create_transport()

Construct the concrete transport for this device.

Returns a :class:~chumicro_deploy.micropython_transport.MicropythonTransport or :class:~chumicro_deploy.circuitpython_transport.CircuitpythonTransport depending on :attr:transport. Deploy-mode translation (ram/flash to the transport's native label) happens here.

When :attr:transport_factory is set, returns its result instead.

Deployer

chumicro_deploy.deployer

Deployer: orchestrate file staging + entrypoint execution on a Device.

The :class:Deployer owns the end-to-end "push code onto a board and run it" flow. It consumes a :class:~chumicro_deploy.sources.FileSource, constructs a transport via :class:~chumicro_deploy.device.Device, stages every file the source returns, executes the entrypoint, and returns a :class:~chumicro_deploy.result.DeployResult.

The transport-level primitive this builds on is :meth:~chumicro_deploy.protocol.TransportProtocol.deploy_files. Per-file iteration and per-group resets are exposed through the richer stage() / execute() flow instead.

Before each deploy, a pre-flight pass auto-promotes a RAM-mode device to flash when the source ships a library marked [tool.chumicro] requires_flash = true. The promotion is announced through on_preflight_message (stderr by default). Pass force_deploy_mode to bypass pre-flight.

Deployer

End-to-end deploy orchestrator.

Parameters:

Name Type Description Default
device Device

Target board configuration. :attr:Device.create_transport is called once per :meth:deploy_diff invocation.

required

device property

The target :class:Device this Deployer was constructed with.

deploy_diff(source, *, clean=True, wipe=False, force_deploy_mode=None, on_progress=None, on_file_staged=None, on_file_deleted=None, on_execute_line=None, on_preflight_message=None, tail_seconds=None)

Diff-deploy source: delete stale in-scope files, then deploy.

  1. Connect.
  2. Ask the transport for every in-scope file currently on the device (list_files_in_scope).
  3. Compute the stale set, the paths on the device that aren't in the new payload.
  4. Delete the stale set (delete_files).
  5. Hand off to the normal :meth:deploy_files for the actual write + execute.

Out-of-scope files (user-uploaded images, hand-edited settings.toml, etc.) are never touched; see :func:chumicro_deploy.protocol.is_in_deploy_scope for the rule.

Mode-aware: in RAM-mode deploys (CP RAM, MP mount) the transport's list_files_in_scope returns an empty list and the diff routine collapses to a plain :meth:deploy_files call, because RAM mode never wrote to flash so there's nothing persistent to diff against.

Parameters:

Name Type Description Default
source FileSource

:class:FileSource to deploy.

required
clean bool

Clean-slate reconcile (the default). The diff scope is the whole device minus the closed keep set (:data:flash_drive.DEVICE_KEEP_SET) + device-managed noise, so a stale board settings.toml / leftover user file is removed and only payload + keep set remain. False is the additive scope (the --no-wipe opt-out): only the entrypoint/state files + /lib are reconciled, other root files preserved. Distinct from wipe: clean-slate keeps the keep set; wipe erases the whole filesystem including it.

True
wipe bool

When True, call transport.wipe_filesystem() before staging, a full destructive erase (keep set included) for corruption-recovery flows. Skips the diff cleanup entirely (nothing left to diff against after a wipe). RAM-mode transports treat the wipe as a no-op, so no need to gate on mode at the call site.

False
force_deploy_mode str | None

Override the pre-flight requires_flash policy. None (default) runs the pre-flight check that auto-promotes RAM → flash when a flagged library is in the graph; "ram" keeps RAM mode even when a library is flagged; "flash" forces flash.

None
on_progress Callable[[float, str], None] | None

Optional (fraction, message) callback. Stages: connecting, listing in-scope / wiping, cleaning stale, staging, executing, done.

None
on_file_staged Callable[[str], None] | None

Forwarded to deploy_files.

None
on_file_deleted Callable[[str], None] | None

Per-file callback invoked with each stale on-device path before deletion.

None
on_execute_line Callable[[str], None] | None

Forwarded to deploy_files.

None
on_preflight_message Callable[[str], None] | None

Optional callback for the "switching to flash mode" message the requires_flash pre-flight emits. Defaults to sys.stderr.

None
tail_seconds float | None

CP-only override for how long the transport captures serial output after the entrypoint's soft-reboot. None keeps the transport's built-in default; ignored on MP transports.

None

Returns:

Type Description
DeployResult

class:DeployResult with success, staged_files,

DeployResult

execute_output, and traceback populated. The

DeployResult

staged_files field carries the new payload's keys

DeployResult

(the deletion list is observable via on_file_deleted

DeployResult

during the call but not retained on the result).

Result types

chumicro_deploy.result

DeployResult and DeployError: outcome types returned by the deployer.

Defined in their own module so callers can annotate against a stable return type without pulling in the orchestration code.

DeployError

Bases: Exception

A deploy step failed in a way the caller should surface directly.

Raised for deploy-specific failures (oversized payload, missing entrypoint, transport staging error after retries exhausted). Transport-level errors propagate as their own types (e.g. :class:~chumicro_deploy.circuitpython_transport.CircuitpythonTransportError) so callers can branch on the underlying cause.

DeployResult dataclass

Outcome of a single Deployer.deploy_diff() call.

Attributes:

Name Type Description
success bool

True when the entrypoint executed and the board reported a clean exit (no traceback, no safe-mode banner).

staged_files list[str]

On-device paths that were written (for flash mode) or the host-side staging paths that fed the RAM-mode inline execution. Empty when staging never completed.

execute_output str

Combined stdout captured from the board while the entrypoint ran. Empty when the execute step never fired.

traceback str | None

Traceback text extracted from execute_output when the board reported one, otherwise None. A non-None traceback implies success=False.

File sources

chumicro_deploy.sources

File sources: plug-in input types that feed the Deployer.

A :class:FileSource produces two things: a mapping of on-device paths to byte contents, and the name of the entrypoint file the runtime should boot into. Three built-ins cover the common cases:

  • :class:FileMapSource when the caller already has an in-memory dict.
  • :class:DirectorySource to ship a directory tree from disk.
  • :class:ImportGraphSource to walk imports from an entrypoint file and ship exactly the transitively-reachable modules.

Custom sources implement the :class:FileSource protocol. The protocol is :func:~typing.runtime_checkable so isinstance works at the deployer boundary.

UnresolvedImportError

Bases: Exception

Raised when an import walk reaches a module that ships nowhere.

:class:ImportGraphSource refuses the deploy at construction time when an import in the bundle resolves to no file under any search path and the module name is not a known device-runtime built-in (see :data:chumicro_deploy.import_allowlist.DEVICE_BUILTIN_MODULES). Such an import would ship silently and raise :exc:ImportError from the device runtime at first boot; refusing here turns a boot-time crash into a deploy-time error that names the importing file and the missing module.

The message leads with "Deploy refused: unresolved import" so :func:chumicro_deploy.recovery.classify_deploy_failure routes it to :attr:~chumicro_deploy.recovery_kind.DeployFailureKind.UNRESOLVED_IMPORT without coupling the classifier to this subclass.

Attributes:

Name Type Description
unresolved

Every (importing_file, module_name) pair the walk collected before refusing. Collect-all-then-fail, so a single deploy attempt surfaces every missing module at once.

FileSource

Bases: Protocol

What the Deployer expects from any file source.

Implementations return the deploy-time file map and declare which of those files is the entrypoint. The Deployer does not mutate either result. Sources own their own state.

files()

Return on-device path → file bytes.

Keys are on-device paths (typically starting with /) that the Deployer writes onto the target. Values are the raw file contents.

entrypoint()

Return the on-device entrypoint path.

Must be one of the keys returned by :meth:files. The Deployer executes this file after staging.

FileMapSource

A source backed by an in-memory dict.

Strings in the input are encoded as UTF-8. Binary values pass through unchanged.

Parameters:

Name Type Description Default
files dict[str, str | bytes]

Mapping of on-device paths to file contents. Values may be str (encoded to UTF-8) or bytes (passed through).

required
entrypoint str

Key in files that is the boot file.

required

Raises:

Type Description
ValueError

If entrypoint is not a key of files.

DirectorySource

A source that ships every file under a host-side directory.

Walks root recursively and reads every file as bytes. The on-device path is the file path relative to root, joined with resource_prefix (default /).

Parameters:

Name Type Description Default
root Path

Host directory whose contents are deployed.

required
entrypoint str

On-device entrypoint path. Must end up as a key in the produced file map, i.e. must be a file under root (relative to resource_prefix).

required
resource_prefix str

On-device prefix prepended to each file's path. Defaults to / (files land at the top of the device filesystem).

'/'
excluded_names frozenset[str] | None

Filename / directory names to skip entirely (exact match, not glob). Defaults to common artifacts (__pycache__, .DS_Store, .git, etc.).

None
target_runtime str | None

When set ("circuitpython" / "micropython" / "cpython"), .py files carrying a __chumicro_runtimes__ marker are skipped if the marker doesn't include target_runtime. None (the default) ships every file. Non-.py files are unaffected.

None

Raises:

Type Description
NotADirectoryError

If root does not exist or is not a directory.

ValueError

If entrypoint is not produced by the walk.

ImportGraphSource

A source that walks Python imports starting from an entrypoint.

Parses entrypoint with the ast module, collects every import and from ... import target, resolves each one against search_paths (first match wins), and recursively walks the resolved modules. Packages are walked through their __init__.py. Dynamic imports (importlib.import_module, __import__) are not detected; pass those names explicitly via extra_modules.

A module name that doesn't resolve against any search path is either a device-runtime built-in (gc, time, board) the host can't provide, or a library the walk should have found but didn't (missing from the search paths, or a typo). The two look identical at the AST level, since both are bare module names. :class:ImportGraphSource distinguishes them by an explicit allowlist: a name on :data:chumicro_deploy.import_allowlist.DEVICE_BUILTIN_MODULES is skipped as a built-in; any other unresolved required import is collected, and the constructor raises :class:UnresolvedImportError naming every importing file and missing module rather than shipping an import that would :exc:ImportError at first boot. Two cases are exempt: an import guarded by try: ... except ImportError (a deliberate optional fallback), and the speculative module.name probe the walk uses to resolve from module import name against a real submodule. When name is a function or class in module/__init__.py instead, the probe doesn't resolve and is silently dropped.

Parameters:

Name Type Description Default
entrypoint Path

Host path to the entrypoint file. Deployed as {device_entrypoint} at the top of the device filesystem.

required
search_paths list[Path]

Directories searched in order when resolving imports. Typical chumicro usage supplies libs/ and packages/.

required
extra_modules list[str] | None

Dotted module names to force-include even when ast cannot see them (dynamic imports, plugin targets).

None
device_entrypoint str

On-device path for the entrypoint (default /code.py).

'/code.py'
resource_prefix str

On-device prefix for every non-entrypoint module file (default /lib).

'/lib'
target_runtime str | None

When set ("circuitpython" / "micropython" / "cpython"), modules carrying a __chumicro_runtimes__ marker for a different runtime are dropped, and their imports are not walked further. None (the default) ships every reachable module, the right setting when the runtime selector reaches both adapters at import time and you want the unmatched one available so the selector can fail loudly on misclassification. The entrypoint itself is never filtered.

None

Honors a module-level __chumicro_skip_factories__ opt-out on entrypoint; see :mod:chumicro_deploy.skip_factories for the marker shape and matching rules. Diagnostics (direct-import overrides, dead-skip notices) surface through :attr:skip_factories_warnings.

Raises:

Type Description
FileNotFoundError

If entrypoint does not exist.

NotADirectoryError

If any of search_paths is not a directory.

ValueError

If __chumicro_skip_factories__ lists entries that match zero discovered factory modules.

UnresolvedImportError

If a required import resolves to no file under any search path and is not a known device built-in. Carries every offending (file, module) pair.

skip_factories_warnings()

Return diagnostic messages emitted by the skip mechanism.

Direct-import overrides and dead-skip notices land here. Empty tuple when the deploy used no skip mechanism or nothing was worth saying.

host_paths()

Return the host filesystem paths every contributed module was read from, including the entrypoint.

Returns a copy so callers can mutate without affecting the source's internal record.

unresolved_imports()

Return every (importing_file, module_name) the walk couldn't ship.

A pair is collected for each required import that resolves to no file under any search path and isn't a known device built-in. A successfully-constructed :class:ImportGraphSource always returns an empty list: a non-empty set aborts construction with :class:UnresolvedImportError (whose unresolved attribute carries the same pairs). The accessor exists so a caller that builds the source under its own try can read the collected pairs off either side.

Probe

chumicro_deploy.probe

probe_device: one-shot runtime-identity probe over a transport.

Thin wrapper over :meth:TransportProtocol.probe_implementation that owns the transport lifecycle (connect / probe / disconnect) and returns a :class:DeviceInfo callers can surface to users.

CPU UID is populated from the probe script itself. :data:~chumicro_deploy.protocol.PROBE_IMPLEMENTATION_SCRIPT reads microcontroller.cpu.uid on CircuitPython and machine.unique_id() on MicroPython and emits it as the fourth field of the __CHU_IMPL__: marker line.

DeviceInfo dataclass

What :func:probe_device gathered from a connected board.

Attributes:

Name Type Description
implementation DeviceImplementation | None

Parsed sys.implementation fields (runtime name, version, machine string). None when the probe script ran but no marker came back, almost always a hardware or connection issue.

board_id str

Normalized board identifier (e.g. "raspberry_pi_pico_w"). Empty string when no board-specific probe has populated it.

uid str

Hex-uppercase CPU UID probed from microcontroller.cpu.uid (CP) or machine.unique_id() (MP) via :data:PROBE_IMPLEMENTATION_SCRIPT. Empty when the probe couldn't read the ID (older firmware, ROM without the hardware ID, or the module raised).

probe_device(device)

Connect, probe, disconnect, and return what the board reports.

Exceptions during the probe propagate. Wrap the call when a soft-failure shape is needed. The transport is always disconnected, even on error.

Parameters:

Name Type Description Default
device Device

Target :class:Device.

required

Returns:

Type Description
DeviceInfo

class:DeviceInfo with the implementation field populated

DeviceInfo

when the board returned the probe marker. uid mirrors

DeviceInfo

implementation.uid when the probe could read it.

Firmware

chumicro_deploy.firmware

Firmware URL resolution + flashing.

Two surfaces: :func:resolve_firmware_url (pure URL formatter) and :func:flash_firmware (download + apply, destructive). See :func:flash_firmware's docstring for the UF2-vs-esptool method selection and per-path recovery strategies.

FlashFirmwareError

Bases: Exception

Raised when a flash step fails.

Message always includes recovery guidance, e.g. "put the board in bootloader mode and retry" or "install esptool first". Catchers typically surface the message directly to the user rather than trying to introspect.

resolve_firmware_url(board_id, runtime, version, *, language=_DEFAULT_LANGUAGE)

Format the firmware download URL for a known board + version.

Pure URL formatter. No network access. Use this when you have an explicit version (e.g. from CI, a release script, or user-supplied). For "give me the latest version available" use :func:firmware_url.derive_firmware_url or :func:firmware_url.latest_circuitpython_url instead.

Parameters:

Name Type Description Default
board_id str

Board identifier. CircuitPython boards use the Adafruit ID (e.g. "raspberry_pi_pico_w", "adafruit_feather_esp32s3_4mbflash_2mbpsram").

required
runtime str

"circuitpython" or "micropython".

required
version str

Firmware version. For CircuitPython, the Adafruit release label ("10.1.4" for stable; "10.2.0-rc.0" for pre-release).

required
language str

Adafruit language code (CircuitPython only). Defaults to "en_US".

_DEFAULT_LANGUAGE

Returns:

Type Description
str

Fully-formed download URL.

Raises:

Type Description
UnresolvedFirmwareError

If runtime is not supported, or if any required field is empty. cause carries the specific failure; see :class:UnresolvedFirmwareError.

flash_firmware(url, device, *, reflash_method=None, bootloader_drive_path=None, interactive=True, erase_flash=True, flash_offset='0x0', on_progress=None)

Download url and flash it onto device.

Destructive: overwrites whatever firmware is currently installed. Progress is reported in rough halves, 0.0 to 0.5 covers download and 0.5 to 1.0 covers flash.

Method selection:

  • "uf2" for RP2040 / RP2350 (Pi Pico family) and any board shipping TinyUF2. Requires a .uf2 URL and writes through the UF2 bootloader drive. Programmatic bootloader entry works on CircuitPython and on MicroPython ports that implement machine.bootloader().
  • "esptool" for ESP32 family boards (ESP32, S2, S3, C3, C6) regardless of runtime. Requires a .bin URL. The caller puts the board in ROM bootloader (the helper handles programmatic entry on boards wired for USB-CDC DTR/RTS, and prompts for a manual BOOT-hold otherwise).

When reflash_method is None, the method is inferred from the URL extension (.uf2 to UF2, .bin to esptool).

Parameters:

Name Type Description Default
url str

Firmware download URL, typically from :func:resolve_firmware_url.

required
device Device

Target :class:Device. device.address addresses the board for both bootloader entry and, on the esptool path, the serial flash.

required
reflash_method str | None

"uf2", "esptool", or None to infer from the URL extension.

None
bootloader_drive_path Path | None

UF2 path only. Skips auto-detection and writes directly to this path.

None
interactive bool

UF2 / esptool path. When True (the default) and programmatic bootloader entry fails, prompts the user to enter bootloader manually. Set False to raise :class:FlashFirmwareError instead.

True
erase_flash bool

esptool path only. When True (the default), wipes user partitions (CIRCUITPY drive, NVS, stored wifi credentials) before writing. Set False to preserve them across an in-place upgrade.

True
flash_offset str

esptool path only. Defaults to "0x0" for CircuitPython's combined image. Use "0x1000" for MicroPython ESP32 / S2 / S3 builds (which ship separate bootloader, partition table, and app sections). Wrong offset writes the app image over the bootloader region and leaves an unbootable chip until a manual reflash.

'0x0'
on_progress Callable[[float, str], None] | None

Optional (fraction, message) callback.

None

Raises:

Type Description
FlashFirmwareError

Download, bootloader entry, drive detection, copy, reboot, or esptool invocation failed. The message names the step and carries recovery guidance.

ValueError

Unknown reflash_method, or reflash_method is None and the URL extension is neither .uf2 nor .bin.

Interactive recovery

chumicro_deploy.recovery

Recovery layer around :class:Deployer.

Transport failures during deploy fall into a small set of classes (port busy, CIRCUITPY drive missing, raw REPL unresponsive, rsync balked mid-copy). Most are recoverable when the user takes a concrete physical action (close the program holding the port, tap RESET, replug USB) and retries. This module classifies a transport exception into a :class:DeployFailureKind, looks up the canned :class:RecoveryPlan for that kind, and wraps :class:Deployer in a coaching loop that surfaces the plan and retries when retryable.

PortHolder dataclass

A process currently holding a serial port open.

Returned by :func:diagnose_port_holders so output to the user can name the process blocking the deploy instead of the generic "close the app holding the port" hint.

Attributes:

Name Type Description
pid int

Process id reported by lsof.

command str

Full command line via ps -o command=. More informative than lsof's short c field for distinguishing stale chumicro processes from an external app explicitly opened by the user.

RecoveringDeployer

:class:Deployer wrapper that classifies failures and coaches recovery.

Two modes selected by the prompt argument:

  • Non-interactive (prompt=None, the default): runs once, prints the classified failure + ordered fix steps on a transport error, then re-raises.
  • Interactive (prompt=input or any callable taking the prompt text and returning the user's reply): runs up to max_attempts times, asks between attempts. An empty / pure- whitespace reply continues; any reply starting with q, a, or e aborts.

On a :class:DeployResult with success=False and a traceback, both modes print the traceback + the TRACEBACK_RETURNED plan and return the result unchanged. A source-level bug isn't something retrying the same bytes will fix.

Parameters:

Name Type Description Default
deployer Deployer

Underlying :class:Deployer that owns the device and transport construction.

required
prompt Callable[[str], str] | None

Callable that asks the user whether to retry. None (default) is non-interactive: report once and re-raise. Pass :func:input (or any (str) -> str callable) for the interactive retry loop.

None
max_attempts int

Ceiling on retry attempts when prompt is set. Ignored in non-interactive mode (always 1 attempt).

_DEFAULT_MAX_ATTEMPTS
output Callable[[str], None]

Injectable output sink. Defaults to :func:print.

print
fskit_wedge_detector Callable[[], bool]

Injectable probe for the macOS FSKit wedge. Promotes CIRCUITPY_DRIVE_MISSING failures to MACOS_FSKIT_WEDGED when the detector fires, so the recovery plan points the user at the right fix.

detect_fskit_wedge

deployer property

The underlying non-recovery :class:Deployer.

deploy_diff(source, *, clean=True, wipe=False, on_progress=None, on_file_staged=None, on_file_deleted=None, on_execute_line=None, tail_seconds=None, force_deploy_mode=None)

Diff-deploy source with classify / coach / (optional retry).

classify_deploy_failure(error)

Map a deploy-path exception to a :class:DeployFailureKind.

The classifier is intentionally string-based: it inspects str(error).lower() against :data:_CLASSIFICATION_TABLE. That keeps the classifier decoupled from the specific exception subclass, which matters because a raised CircuitpythonTransportError often wraps a SerialException or OSError whose text is the real signal.

Returns :attr:DeployFailureKind.UNKNOWN when no row matches.

diagnose_port_holders(port_path, *, runner=subprocess.run)

Return processes currently holding port_path open.

Uses lsof -F pcn <port> on POSIX. The output format is one field per line tagged with a single character (p for PID, c for short command, n for path). Each PID record starts with a p line, and subsequent c / n lines belong to it until the next p or EOF.

Returns an empty list on Windows (no portable equivalent without handle.exe), when lsof isn't installed, when the port is not held, or on any subprocess failure. The caller treats absence of holders as "we couldn't tell, fall through to the generic recovery hint" rather than an error.

A deploy interrupted by Ctrl-C that leaves an orphan chumicro-deploy / mpremote subprocess holding the port surfaces as "failed to access ... it may be in use by another program". Showing the PID lets the user kill the right thing without running lsof themselves.

recovery_plan_for(kind)

Return the canned :class:RecoveryPlan for kind.

macOS FSKit wedge detection

chumicro_deploy.macos_fskit

macOS FSKit / DiskArbitration wedge detection.

Recent macOS releases moved the FAT (msdosfs) driver out of the kernel and into a user-space FSKit extension (com.apple.fskit.msdos.appex). When that extension hits an internal error mid-probe (most often on small CIRCUITPY FAT12 volumes), it can leave diskarbitrationd stuck in an uninterruptible kernel wait. Symptoms:

  • diskutil list hangs indefinitely.
  • Newly inserted CIRCUITPY drives never appear under /Volumes.
  • Unplug / replug of the board does nothing; the bus enumerates fine (system_profiler SPUSBDataType shows the device) but no mount ever happens.

The cheapest reliable signal is the state column of diskarbitrationd: a healthy daemon sits in Ss (interruptible sleep), a wedged one sits in Us / U* (uninterruptible wait). :func:detect_fskit_wedge reads that state via pgrep + ps and reports the wedge.

Recovery requires sudo (killing system daemons), so this module does not auto-run it. It just surfaces the exact command in :data:MACOS_FSKIT_RECOVERY_COMMAND for the user-facing coaching output to paste verbatim.

detect_fskit_wedge(*, runner=subprocess.run, platform=_sys_module.platform)

Return True when macOS diskarbitrationd is wedged.

Always returns False on non-macOS platforms, because the FSKit rewrite is Apple-only, so the wedge mode does not exist elsewhere.

Implementation:

  1. pgrep diskarbitrationd to find the PID. No PID (daemon missing or in the process of respawning) means not wedged.
  2. ps -o state= -p <pid> reads the process state column. A state containing U means uninterruptible kernel wait, which is the wedge signature. Healthy daemons sit in S (interruptible sleep).

Both subprocess calls get short timeouts, because a wedged system can stall other commands too, and we don't want detection itself to hang.

Parameters:

Name Type Description Default
runner Callable[..., CompletedProcess]

Injectable subprocess runner. Must match the subprocess.run signature closely enough for capture_output=True, text=True, check=False, timeout=… to work.

run
platform str

Injectable sys.platform value.

platform

Returns:

Type Description
bool

True if the daemon is in uninterruptible wait, else

bool

False. Any subprocess failure (missing pgrep/ps,

bool

timeout, non-zero exit) is treated as "not wedged" so a

bool

false positive can never block a legitimate retry.

Host platform compatibility

chumicro_deploy.host_platform

Host-platform compatibility checks for chumicro-deploy.

The deploy package targets developer laptops: fully supported on macOS, fully viable on Linux, and explicitly unsupported on native Windows. This module surfaces those gaps as fast-failing errors with actionable remediation hints rather than letting them manifest as no devices found deep inside the deploy flow.

Two checks live here:

  • :func:check_supported_platform, called before any path that enumerates serial ports or USB mount points, so Windows users see a "use WSL2" message immediately instead of an empty-port-list dead end.
  • :func:check_rsync_available, called before flash-mode deploy. The CircuitPython flash transport shells out to rsync and there is no fallback (FAT32 timestamp semantics make shutil.copytree unreliable for incremental writes). Detecting absence up-front beats failing partway through staging.

WindowsNotSupportedError

Bases: RuntimeError

Raised when chumicro-deploy is invoked on native Windows.

RsyncMissingError

Bases: RuntimeError

Raised when rsync is required for flash-mode deploy but is not on PATH.

is_native_windows()

Return True on native Windows (win32 / cygwin).

WSL2 reports sys.platform == "linux" and is treated as Linux throughout the deploy code, which is correct since the POSIX paths work inside WSL2.

check_supported_platform()

Raise :class:WindowsNotSupportedError on native Windows.

Call before any path that enumerates serial ports or USB mount points, so users see one clean error before the deploy flow tries to walk POSIX-only paths. No-op on macOS, Linux, and WSL2.

install_hint_for_rsync()

Return a platform-specific install command for rsync.

Detects common Linux package managers via :func:shutil.which and produces a copy-paste-able sudo <manager> install rsync line. Falls back to a generic message when no recognized manager is on PATH. On macOS the system ships rsync, so the hint points at Xcode Command Line Tools / Homebrew as the recovery path.

check_rsync_available()

Raise :class:RsyncMissingError if rsync is not on PATH.

Call before a CircuitPython flash-mode deploy so a missing rsync surfaces as one clear error before the transport opens a serial connection and starts staging. The error embeds the install hint from :func:install_hint_for_rsync.

Devices.yml schema and loader registry

The chumicro_deploy.config package owns the devices.yml schema. load_devices_yml is the built-in loader (registered under the "default" entry-point name); third parties register their own config formats via the chumicro_deploy.config_loaders entry-point group, and discover_config_loaders collects every registered loader keyed by name.

chumicro_deploy.config

Opt-in configuration loaders for :class:~chumicro_deploy.Device.

chumicro-deploy ships one built-in loader (registered as "default") that reads the devices.yml schema defined in :mod:chumicro_deploy.config.default. Third parties with a different config format register a loader by declaring a Python entry point in the chumicro_deploy.config_loaders group. A template repo's pyproject.toml looks like:

.. code-block:: toml

[project.entry-points."chumicro_deploy.config_loaders"]
myformat = "my_pkg.loader:load"

Where my_pkg.loader.load(path, *, device_id=None) returns a :class:Device.

:func:discover_config_loaders collects the built-in default loader plus every registered third-party loader into a single {name: callable} mapping that downstream code can dispatch from.

discover_config_loaders()

Return the registered config-loader callables keyed by name.

The default entry is always present (built-in). Third-party entries come from Python entry points in the chumicro_deploy.config_loaders group. Any name collision with default is rejected by the registry so built-in behavior is not silently shadowed.

Returns:

Type Description
dict[str, ConfigLoader]

{name: loader} mapping where loader(path, *, device_id=None)

dict[str, ConfigLoader]

returns a :class:Device.

chumicro_deploy.config.default

Default YAML config loader that ships with chumicro-deploy.

Loads the devices.yml schema that chumicro-deploy defines and owns. Any project (its own workspace, a project-workspace template, or a third-party consumer) can write this shape to configure its deploy targets without depending on any upstream tooling.

The schema (stable subset that the loader accepts):

.. code-block:: yaml

defaults:
  micropython: <id of default MP device>
  circuitpython: <id of default CP device>
  deploy_mode: ram
devices:
  - id: my-pico-w-mp
    runtime: micropython          # → Device.transport
    address: /dev/cu.usbmodem213101
    serial_baudrate: 115200
    deploy_mode: ram              # optional; falls back to defaults.deploy_mode
    # description and other keys are tolerated
    # but ignored by this loader.

Required fields per device: id, runtime, address. Everything else is optional. Extra keys are silently ignored so consumers that carry additional metadata (test-orchestration hints, setup commands, descriptions) can share the same file.

Importing this module pulls in PyYAML (already a dependency of :mod:chumicro-deploy), so the import cost is paid only when devices.yml is actually used.

Third parties that need a different shape (JSON, TOML, a custom YAML layout) can register their own loader via the chumicro_deploy.config_loaders entry-point group; see :func:chumicro_deploy.config.discover_config_loaders.

DeviceConfigError

Bases: Exception

Raised when a devices.yml is missing or fails validation.

DeviceEntry dataclass

A single device from a validated devices.yml registry.

DeviceDefaults dataclass

Top-level defaults from the defaults: section of devices.yml.

Controls which devices are targeted from IDE play buttons, the default deploy mode for all devices, and which runtimes the IDE targets.

load_device_registry(path=None, *, workspace_root=None)

Load and validate devices.yml into (entries, defaults).

Each entry is a :class:DeviceEntry carrying the description and any extra keys from the YAML alongside the deploy-relevant fields.

Parameters:

Name Type Description Default
path Path | None

Explicit path to devices.yml. When None, falls back to workspace_root / devices.yml.

None
workspace_root Path | None

Directory that contains devices.yml. Defaults to :func:Path.cwd.

None

Returns:

Type Description
tuple[list[DeviceEntry], DeviceDefaults]

Tuple of (devices, defaults).

Raises:

Type Description
DeviceConfigError

File missing, malformed, or fails validation.

load_devices(path=None, *, workspace_root=None)

Load just the validated device list from devices.yml.

Convenience wrapper around :func:load_device_registry.

resolve_ide_devices(devices, defaults)

Return the devices an IDE play button should target.

For each runtime selected by defaults.ide_runtime, picks the device whose ID matches defaults.<runtime>, or the first device of that runtime when no default is set.

load_raw_entries(path)

Parse a devices.yml into raw entries + defaults dict.

Reads the YAML and returns the devices: list and the defaults: mapping verbatim: no field validation, no :class:Device construction, no normalization. Richer schemas layer their own validation on top of this, so the on-disk shape is defined in one place.

Parameters:

Name Type Description Default
path Path | str

Filesystem path to the YAML file.

required

Returns:

Type Description
list[dict[str, Any]]

Tuple of (entries, defaults). entries is the

dict[str, Any]

devices: list verbatim (empty list when the key is

tuple[list[dict[str, Any]], dict[str, Any]]

missing). defaults is the defaults: mapping verbatim

tuple[list[dict[str, Any]], dict[str, Any]]

(empty dict when missing).

Raises:

Type Description
FileNotFoundError

The YAML file does not exist.

ValueError

The YAML root is not a mapping, or defaults: / devices: are present but the wrong type.

load_devices_yml(path, *, device_id=None, runtime=None)

Load one device from a devices.yml file.

Resolution precedence:

  1. device_id wins outright.
  2. runtime picks defaults.<runtime> when device_id is None, so a single-runtime caller disambiguates a two-runtime defaults: block without naming the device id.
  3. Single-default fallback: when both device_id and runtime are None, exactly one runtime default in the file picks itself, otherwise raises.

device_id and runtime are mutually exclusive: passing both raises so the caller cannot accidentally override a specific id with a runtime hint.

Parameters:

Name Type Description Default
path Path | str

Filesystem path to the YAML file.

required
device_id str | None

Which entry to return.

None
runtime str | None

One of "circuitpython" or "micropython".

None

Returns:

Name Type Description
A Device

class:Device corresponding to the selected entry.

Raises:

Type Description
FileNotFoundError

The YAML file does not exist.

ValueError

The file has no matching device, the device_id is not among the entries, or runtime is not one of the supported names.

load_devices_yml_raw(path, *, device_id=None, runtime=None)

Resolve one device and return its raw devices.yml entry.

Identical resolution to :func:load_devices_yml (same precedence, same errors) but returns the unmapped entry dict, including hardware and other keys the :class:Device dataclass drops. Use when fields off the schema are needed alongside the transport facade (firmware-URL derivation reads hardware.firmware_source).

Transport protocol

chumicro_deploy.protocol

Explicit transport protocol for ChuMicro device transports.

Captures the duck-typed transport contract that :class:CircuitpythonTransport and :class:MicropythonTransport both satisfy, in a form type checkers can enforce.

Two protocols are defined:

  • :class:TransportProtocol is the minimum every transport must implement: connect, stage, execute, soft_reset, reset, recover, disconnect.
  • :class:ExtendedTransportProtocol adds the CircuitPython-specific RAM-mode chunking helpers (execute_scripts, probe_free_memory, inline_script_budget_bytes). MicropythonTransport does not need these, because there is no per-script RAM budget on mpremote.

CPython-only: these protocols ride on typing.Protocol, which only the workbench tooling uses (the cross-runtime device libraries that satisfy the contract don't import this module).

Runtime

Bases: StrEnum

Supported device runtime identifiers.

Values round-trip as plain strings so Device(transport="circuitpython") and Device(transport=Runtime.CIRCUITPYTHON) are interchangeable. CLI argparse choices, config-file loaders, and third-party callers should reference members of this enum rather than string literals to keep the allowed set in one place.

DeployMode

Bases: StrEnum

User-facing deploy-mode preference.

RAM keeps edits off the board's flash (inline exec on CircuitPython, mount on MicroPython). FLASH writes files persistently (CIRCUITPY drive copy on CircuitPython, copy on MicroPython). The transport-internal mount-label mapping (rammount, flashcopy) is handled in :meth:~chumicro_deploy.device.Device.create_transport.

ReflashMethod

Bases: StrEnum

Firmware reflash backend selection.

UF2 drives the UF2 bootloader drive path (Pi Pico family, TinyUF2 boards). ESPTOOL shells out to esptool over serial for ESP32-family boards. See :func:~chumicro_deploy.firmware.flash_firmware for the method selection guide.

DeviceImplementation dataclass

Runtime identity probed from a connected board.

Populated by probe_implementation on transports that support it. Identifies the runtime + board behind a connected device, and pins a UID that disambiguates two boards of the same model, so a mounted CIRCUITPY drive can be matched to its connected board without a mount-order-dependent path.

Attributes:

Name Type Description
name str

sys.implementation.name, either "circuitpython" or "micropython".

version str

Dotted version from sys.implementation.version (e.g. "10.1.4" or "1.26.0").

machine str

sys.implementation._machine on both runtimes, a free-form string describing the board (e.g. "Raspberry Pi Pico W with rp2040"). Empty when the firmware does not expose it.

uid str

Hex-uppercase CPU / module unique ID, sourced from microcontroller.cpu.uid on CircuitPython and machine.unique_id() on MicroPython. Matches the UID:... line CircuitPython writes to boot_out.txt on mount, so the host can disambiguate two identical boards that would otherwise share a machine string. Empty on firmware too old to expose the probe path or if the probe itself raised.

UnsupportedExtraFilesError

Bases: NotImplementedError

Raised when transport.stage(extra_files=...) can't be honored.

Currently raised only by CircuitPython RAM mode: there is no writable device-side filesystem to land bytes on (RAM mode runs inline-execed source via raw REPL; CIRCUITPY is host-write but a host write while the device is running can trigger a soft reset that wipes the in-memory state). Staging an on-device file artifact (typically runtime_config.msgpack for chumicro_config.load_runtime_config()) requires flash mode.

TimeSource

Bases: Protocol

Structural interface for an injectable time source.

FakeTime from :mod:chumicro_deploy.testing is one conforming implementation that eliminates wall-clock waits.

DeviceTransportError

Bases: Exception

Base for both runtimes' transport errors.

Orchestration that treats the transports uniformly (the RecoveringDeployer retry loop, the failure classifier) catches this instead of enumerating per-runtime subclasses, so adding a runtime doesn't mean auditing every catch site.

MidDeployDisconnected

Bases: DeviceTransportError

Base for the per-runtime "device dropped mid-deploy" errors.

A distinct branch so callers can except "the cable came out" without conflating it with other transport errors. The original :class:OSError is attached as :attr:cause so callers that need the underlying errno can read it without re-parsing :func:str of the wrapper.

TransportProtocol

Bases: Protocol

Minimum transport contract every device transport must satisfy.

connect()

Verify the device is reachable.

stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)

Prepare the host-side staging area and (mode-dependent) push to device.

extra_modules are additional Python files that the test files import as top-level modules. Each is registered as importable on the device alongside library sources: in RAM mode they join staged_sources so the inline bootstrap registers them; in flash / copy / mount modes they land at the device root next to the test files.

extra_files are non-Python files to land at named device paths. Keys are absolute device paths ("/runtime_config.msgpack"); values are the bytes to write. Typical use is staging a runtime_config.msgpack alongside test files so on-device code can call :func:chumicro_config.load_runtime_config. Per-mode semantics: flash and copy modes write each file to the device's filesystem alongside library + test sources. Mount mode writes to the host directory mounted as the device filesystem. RAM mode raises :class:UnsupportedExtraFilesError because it has no writable device-side filesystem to land bytes on.

execute(bootstrap_script, *, on_line=None)

Run bootstrap_script on the device and return captured stdout.

When on_line is provided, each captured stdout line is dispatched to the callback as it arrives over the serial link, before :meth:execute returns. Lines are dispatched without their trailing newline; \r\n is normalized to \n. When on_line is None (the default), behaviour is the request/response shape every existing call site assumes, and captured stdout still comes back as the return value.

Streaming dispatch lets a host-side consumer react mid-execute: for example, a test fixture that opens a TCP connection only once the board prints a "server ready" line, or a long-running bake harness that surfaces board output as it lands instead of after the bootstrap finishes.

run_script(script, *, timeout=10.0)

Run script on the device without staging and return captured stdout.

Sibling of :meth:execute for short, self-contained scripts that don't need any staged sources / test files / harness: e.g. a feature-detection probe (try: import esp32 etc.), a quick sys.implementation query, an inline diagnostic. Bypasses the stage()-must-be-called-first precondition that :meth:execute enforces.

Parameters:

Name Type Description Default
script str

Self-contained Python source to run via the raw REPL. No imports of staged modules, only built-ins and runtime-provided modules.

required
timeout float

Idle-timeout for the script's output, in seconds.

10.0

Returns:

Type Description
str

Captured stdout from the device.

soft_reset()

Soft-reset the interpreter to clear modules and free heap.

recover()

Aggressive reset after a failure when board state is unknown.

disconnect()

Release transport resources (serial port, mounts, staging dir).

probe_implementation()

Query sys.implementation on the board for PR-summary metadata.

reset_into_bootloader()

Try to put the board into its UF2 bootloader via the running runtime.

Issues a runtime-specific reset command (machine.bootloader() on MicroPython, microcontroller.on_next_reset(RunMode.BOOTLOADER) + microcontroller.reset() on CircuitPython) and swallows the connection-drop that follows. The serial link is torn down as the board resets, so a clean response is not expected.

Returns:

Type Description
bool

True when the command was dispatched (the board

bool

should be rebooting into its bootloader).

bool

False when the runtime does not expose a

bool

bootloader-entry API, the transport could not be opened,

bool

or the command failed before reaching the board.

deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, **transport_kwargs)

Write files onto the device and execute the entrypoint.

Distinct from :meth:stage + :meth:execute. The former pair takes dirs + test files + harness source; this method takes a generic path-to-bytes map and a single entrypoint path.

Parameters:

Name Type Description Default
files dict[str, bytes]

On-device-path -> file-bytes mapping. Paths may start with /; transports normalize as needed.

required
entrypoint str

On-device path (must be a key of files) for the runtime to execute after staging.

required
on_file_staged Callable[[str], None] | None

Optional per-file callback invoked with the on-device path as each file is written.

None
on_execute_line Callable[[str], None] | None

Optional callback invoked once per line of captured execute output, in order. Not guaranteed to stream live, since transports may call it after execute() completes.

None

Returns:

Type Description
str

Combined stdout from the entrypoint execution.

list_files_in_scope(*, clean_slate=False)

Enumerate device files within the deploy's managed scope.

Returns the file set on the device today that the next deploy would replace. Files in this set that aren't in the next deploy's payload are the stale set to delete before writing.

clean_slate=True (the deploy default) widens the scope to the whole device minus the closed keep set (:data:flash_drive.DEVICE_KEEP_SET) and device-managed noise, so a stale board settings.toml or leftover user file is reconciled away. clean_slate=False is the additive scope (the --no-wipe opt-out): only the entrypoint / state files and the library tree, leaving other root files in place.

Returns paths in the same leading-slash form :meth:deploy_files accepts ("/lib/foo.py", "/code.py", "/active.py", etc.). Order is unspecified; callers sort if they need deterministic output.

Transports that don't support persistent state (RAM-mode deploys: nothing survives across deploys to be diffed) return an empty list.

delete_files(paths)

Delete paths from the device's filesystem.

No-op when paths is empty. Each path must be in the same leading-slash form :meth:deploy_files accepts; transports normalize internally. Missing paths are tolerated silently: a previous deploy may have already removed something, so re-deleting shouldn't error.

Best-effort: a single delete failure logs a warning but does not abort the batch. The deploy that follows still writes the new payload, and leaving a stale file in scope is preferable to skipping the deploy outright.

clear_entrypoints()

Delete any persisted code.py / main.py and confirm the deletion landed on the device before returning.

Call once before a soft-reset when a stale entrypoint from a prior deploy would otherwise run on the reboot. Filesystem transports unlink the files and verify they are gone, flushing FAT first on CIRCUITPY so the bytes hit the medium before the reboot re-reads it. RAM and mount transports never persist an entrypoint, so this is a no-op there.

wipe_filesystem()

Erase the device's user filesystem before the next deploy.

Destructive: wipes every file the runtime can see, both in-scope (/lib/*, /code.py / /main.py / etc.) and out-of-scope (/settings.toml, hand-edited boot.py, user-uploaded assets). Use for clean-slate / corruption- recovery flows where an ordinary diff-deploy isn't enough, and to reclaim flash filled with stage residue that has hit ENOSPC mid-deploy.

Per-runtime recipe matrix:

  • CircuitPython (any board): import storage; storage.erase_filesystem() over the raw REPL. Reformats the FAT volume and triggers a hard reset; the transport swallows the USB-CDC drop and re-establishes raw REPL after the volume re-mounts.
  • MicroPython on rp2 (Pi Pico W et al.): os.umount('/'); os.VfsLfs2.mkfs(rp2.Flash()); machine.soft_reset().
  • MicroPython on esp32 (Lolin S2 family et al.): os.umount('/'); os.VfsLfs2.mkfs(esp32.Partition.find( TYPE_DATA, label='vfs')[0]); machine.soft_reset().
  • Other MicroPython substrates: raise RuntimeError until a verified recipe is added.

mkfs is required for MicroPython rather than a recursive os.remove walk: LittleFS metadata + wear-leveling artifacts survive a file-by-file delete, so a board with a small partition (rp2's ~850 KB, esp32's 2 MB) can still hit ENOSPC mid-deploy after a non-mkfs "wipe." Firmware partitions are untouched on every runtime.

RAM-mode / mount-mode deploys (CP RAM, MP mount) are no-ops, since neither writes to flash, so there's nothing persistent to wipe. No need to gate on mode at the call site.

ExtendedTransportProtocol

Bases: TransportProtocol, Protocol

Transport contract plus the CircuitPython RAM-mode chunking helpers.

Implemented by :class:CircuitpythonTransport. Runtime-check against this protocol before calling the chunked-execute helpers.

execute_scripts(bootstrap_scripts, *, on_line=None)

Run multiple bootstrap scripts in one interpreter session.

on_line threads through each per-script :meth:TransportProtocol.execute call, so a chunked CP RAM-mode deploy dispatches stdout lines across all scripts in arrival order under a single callback.

probe_free_memory()

Return free heap bytes reported by the connected board.

inline_script_budget_bytes()

Return a conservative per-script budget based on live free heap.

is_in_deploy_scope(device_path)

Return True when device_path falls inside the deploy's managed scope.

Scope rule:

  • The four entrypoint / state files in :data:DEPLOY_SCOPE_FILES (/code.py, /main.py, /active.py, /runtime_config.msgpack).
  • Everything under /lib/; see :data:DEPLOY_SCOPE_PREFIXES.

Anything else (user-uploaded images, manually-edited boot.py overrides, hand-tuned settings.toml knobs) is out of scope and survives every diff-deploy untouched.

The check is path-shape only: callers should normalize their paths to leading-slash form before calling.

validate_entrypoint_in_files(files, entrypoint, *, error_cls=ValueError)

Raise error_cls if entrypoint is not a key of files.

The exact message text ("entrypoint <name> missing from files ...") is pattern-matched by :func:~chumicro_deploy.recovery.classify_deploy_failure to route to :attr:DeployFailureKind.CONFIGURATION_ERROR, so the message shape is part of the helper's contract.

parse_probe_output(output)

Extract a :class:DeviceImplementation from probe stdout.

Scans for the __CHU_IMPL__: marker line emitted by :data:PROBE_IMPLEMENTATION_SCRIPT and ignores any surrounding output. When an accompanying __CHU_UID__: line is present, its hex payload is attached to :attr:DeviceImplementation.uid; output without that line parses cleanly with uid="". Returns None when the __CHU_IMPL__: marker is missing or its payload is malformed (the "probe unavailable" signal).

Parameters:

Name Type Description Default
output str

Combined stdout (and stderr if merged) from the probe script's execute call.

required

write_files_to_staging(staging_path, files, on_file_staged=None)

Write a device-path → bytes mapping into a host staging tree.

The shared front half of both transports' deploy_files: sorted for deterministic staging order, leading slashes stripped so the device-path → staging-path translation is reversible, parent dirs created on demand, per-file callback fired after each write.

MicroPython transport

chumicro_deploy.micropython_transport

MicroPython device transport using mpremote.

Two execution paths:

  • Persistent serial transport (mount mode and per-execute path): uses mpremote.transport_serial.SerialTransport directly. Opens the serial port once per session, enters raw REPL once, mounts the staging directory once, and runs each bootstrap via exec_raw. Avoids the cold-start cost of spawning mpremote per execute() call.
  • Subprocess fallback (copy mode staging, reset/recover): uses the mpremote CLI for operations that are one-shot and easier to express via the CLI. The serial port is closed before these calls and reopened on the next execute().

MicropythonTransportError

Bases: DeviceTransportError

Raised when an mpremote command fails.

MicropythonMidDeployDisconnected

Bases: MicropythonTransportError, MidDeployDisconnected

Raised when the device drops mid-deploy.

The MicroPython face of :class:.protocol.MidDeployDisconnected (which owns the message shape and the :attr:cause attribute).

MicropythonTransport

Transport for MicroPython boards.

Parameters:

Name Type Description Default
address str

Serial port or network address of the device.

required
baudrate int

Serial baud rate (default 115200). Only used for the persistent serial transport, because subprocess mpremote invocations negotiate baud rate themselves.

115200
mode str

"mount" (default) or "copy".

'mount'
runner Callable[..., CompletedProcess] | None

Callable that executes subprocess commands. Accepts the same signature as subprocess.run. Defaults to subprocess.run.

None
transport_factory Callable[[str, int], Any] | None

Callable that builds a :class:SerialTransport given (address, baudrate). Defaults to :func:_default_transport_factory. Inject a fake to avoid opening a real serial port.

None
time TimeSource | None

Object providing monotonic() and sleep(). Defaults to Python's time module. Inject :class:FakeTime (from :mod:chumicro_deploy.testing) to eliminate wall-clock waits.

None

connect()

Verify the device is reachable by running a no-op command.

Uses subprocess so the persistent serial transport is opened lazily on the first execute() (or eagerly during mount-mode stage()). Avoids holding the serial port during the gap between connect() and stage().

stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)

Prepare a staging directory with library sources, tests, and harness.

In mount mode, the staging directory is mounted on the device via the persistent serial transport. In copy mode, it is recursively copied to flash via mpremote fs cp -r.

Safe to call repeatedly on the same transport (RAM-mode orchestration re-stages per file). On re-stage in mount mode the existing mount is dropped first. Otherwise mpremote's mount_local hits OSError: [Errno 1] EPERM because the device-side mount hook refuses to replace a live mount.

Parameters:

Name Type Description Default
source_dirs list[Path]

Library src/ directories to include.

required
test_files list[Path]

Test files to stage.

required
harness_source Path

Path to the test harness src/ directory.

required
extra_modules list[Path] | None

Optional sibling Python files to copy to the staging root next to the test files.

None
extra_files dict[str, bytes] | None

Non-Python files to land at named device paths (typically {"/runtime_config.msgpack": <bytes>}) so test code can call chumicro_config.load_runtime_config(). Copy mode only. Copy mode rsyncs the staging tree to flash, so a file staged for /runtime_config.msgpack lands at that absolute path. Mount mode raises :class:UnsupportedExtraFilesError: mpremote mount_local lands the staging tree at /remote, so the bytes would appear at /remote/runtime_config.msgpack, invisible to a reader that opens the absolute /runtime_config.msgpack.

None

Raises:

Type Description
UnsupportedExtraFilesError

mode == "mount" and extra_files is non-empty. Switch the device's deploy_mode to "copy" to fix.

execute(bootstrap_script, *, on_line=None)

Execute a bootstrap script on the device and return captured output.

Uses the persistent serial transport's exec_raw so each call amortizes the one-time mpremote-cold-start cost. In copy mode the serial transport is opened lazily (since stage() released it for the fs cp subprocess).

Parameters:

Name Type Description Default
bootstrap_script str

Python code to execute on the device.

required
on_line Callable[[str], None] | None

Optional callback invoked with each stdout line as it arrives over the serial link, before this method returns. Lines are delivered without their trailing newline; \r\n is normalized to \n. Wires straight into mpremote's data_consumer hook on exec_raw, which fires per byte during the stdout read phase and stops at the \x04 end-of-stdout marker. The dispatcher discards that terminator byte so callbacks only see board content.

None

Returns:

Type Description
str

Captured stdout from the device.

run_script(script, *, timeout=10.0)

Run script via the persistent serial REPL with no staging.

Opens the raw REPL if needed and runs the script directly via exec_raw. Bypasses the stage()-first precondition that :meth:execute enforces because script is expected to be self-contained (no imports of staged modules).

Parameters:

Name Type Description Default
script str

Self-contained Python source to run.

required
timeout float

Idle-timeout for the script's output, in seconds.

10.0

Returns:

Type Description
str

Captured stdout from the device.

soft_reset()

Soft-reset the device to clear interpreter state.

Sends Ctrl-D through the persistent raw REPL. This clears sys.modules and the device heap without toggling USB, so it is safe to run between every file.

The mount is not restored by this method. Callers always call stage() next, and stage() owns the fresh mount. Restoring the mount here would double-wrap mpremote's SerialIntercept (mount_local unconditionally wraps self.serial) and garble subsequent I/O.

Leaves the interpreter clean and the serial un-wrapped. If no persistent transport is open, falls back to a subprocess mpremote reset with a bounded settle-and-retry: a prior hard failure (e.g. a stage ENOSPC) can wedge the board so its USB-CDC re-enumerates, and the first mpremote connect ... reset then fails it may be in use by another program. Retrying across that window keeps one library's failure from cascading through every subsequent per-library reset.

recover()

Attempt to recover after a failed test.

Closes the persistent transport (if open) and reconnects from scratch. More aggressive than :meth:soft_reset because the previous failure might have left the raw REPL in an unknown state where exit_raw_repl itself could hang.

reset_into_bootloader()

Issue machine.bootloader() to drop into the UF2 bootloader.

Whether the call actually puts the chip in bootloader mode depends on the board's MicroPython build. RP2040 / RP2350 ports wire machine.bootloader() to the native UF2 ROM bootloader and it just works. Other ports may not implement the function at all, or may implement it as a plain reset that boots straight back into the app. We try blindly rather than maintain a board / firmware-version table: a successful entry surfaces as a new ROM-bootloader serial port visible to whatever polls for one, and an unsuccessful entry surfaces as no-new-port. Don't add board-specific branching here: firing machine.bootloader() blindly and letting the caller's drive-poll be the success signal handles every port the same way.

probe_implementation()

Query sys.implementation on the board for PR-summary metadata.

Opens the persistent raw REPL if needed and runs a short inline script. No staging required because sys.implementation is a built-in attribute. Failures are swallowed (None returned) so a flaky or unusual firmware never blocks the real test run.

Returns:

Type Description
DeviceImplementation | None

class:DeviceImplementation on success, or None if the

DeviceImplementation | None

probe could not complete or its output did not contain the

DeviceImplementation | None

expected marker line.

disconnect()

Clean up staging directory and close the persistent serial transport.

deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, follow='exec', clean=False)

Write files onto the device and execute entrypoint.

In mount mode, the file tree is staged host-side, mounted through the persistent raw REPL, and the entrypoint runs from the mount point. In copy mode, the staging tree is pushed to flash via mpremote fs cp -r and the entrypoint runs in whichever follow mode follow selects.

Parameters:

Name Type Description Default
files dict[str, bytes]

On-device-path -> bytes mapping. Leading slashes are stripped before joining with the staging dir or device root.

required
entrypoint str

On-device path, must be a key of files.

required
on_file_staged Callable[[str], None] | None

Per-file callback invoked with the on-device path as each file is written to staging.

None
on_execute_line Callable[[str], None] | None

Callback invoked once per output line, in order, after execution completes.

None
follow Literal['exec', 'soft_reboot']

Selects how the entrypoint runs.

"exec" (default) wraps the entrypoint in an exec(open(...).read()) bootstrap and runs it via self._serial.exec_raw with an inter-byte idle timeout. Captures full stdout once the script returns and the raw REPL emits its EOF (\x04) marker. Right for return-bounded scripts, hangs forever on while True bodies.

"soft_reboot" sends Ctrl-B + Ctrl-D and lets MicroPython auto-run /main.py, then reads serial output up to :attr:timeout seconds (returning partial output for infinite-loop bodies). Requires copy mode with /main.py as the entrypoint, since MicroPython only auto-runs boot paths on flash.

'exec'
clean bool

Copy mode only. When True, wipe the device's /lib tree before pushing. Top-level user files (boot.py, main.py, settings.toml) live outside /lib and survive.

False

Returns:

Type Description
str

Combined stdout captured from the entrypoint execution.

str

For follow="soft_reboot" and an infinite-loop body,

str

this is whatever accumulated up to self.timeout.

Raises:

Type Description
MicropythonTransportError

follow="soft_reboot" was requested with mount mode or with an entrypoint other than /main.py.

list_files_in_scope(*, clean_slate=False)

Implements the contract on :meth:TransportProtocol.list_files_in_scope.

Copy mode dispatches the on-device walk via raw REPL (:data:._device_scripts.LIST_ALL_SCRIPT for clean_slate=True, :data:._device_scripts.LIST_SCOPE_SCRIPT otherwise). Mount mode (RAM) returns an empty list.

Raises :class:MicropythonTransportError on raw-REPL failure.

delete_files(paths)

Delete paths and reap directories that become empty.

No-op in mount (RAM) mode, because mount mode doesn't write to flash, so there's nothing to delete. Otherwise sends a small script that calls os.remove on each path then walks the whole filesystem bottom-up and os.rmdir-reaps any directory that's now empty; missing paths and per-path errors are tolerated silently so the diff-cleanup pass never blocks the actual deploy.

The reap is load-bearing on MicroPython: os.remove drops files but never their parent directory, and MP's default sys.path is ['', '.frozen', '/lib'] with '' (root) FIRST. An empty root-level /<pkg>/ resolves import <pkg> to a PEP 420 namespace package (__file__ is None, dir() empty) and shadows the populated /lib/<pkg>/ deeper in the path. rmdir only removes an empty directory so live packages are never touched. Dot-prefixed entries are skipped: they're not ours to reap.

clear_entrypoints()

Remove main.py / code.py and confirm they are gone.

Copy mode only. Mount (RAM) mode never writes a persistent entrypoint. Runs os.remove on the device (authoritative immediately, since unlike the CIRCUITPY host-FAT path there is no flush lag) and, in the same raw-REPL script, re-stats each path so a still-present entrypoint raises on the device and surfaces as a loud error. Call once before a soft-reset so the reboot cannot race a stale entrypoint left by a prior deploy.

wipe_filesystem()

Implements the contract on :meth:TransportProtocol.wipe_filesystem.

Dispatches :data:_WIPE_FILESYSTEM_SCRIPT as a one-shot mpremote exec subprocess rather than over the persistent :class:SerialTransport: machine.soft_reset() clears interpreter state and the raw_repl framing, so reusing the persistent transport across the reset would leave it in an inconsistent state. After the subprocess returns, the serial port is given :data:_WIPE_REBOOT_SETTLE_SECONDS to settle so the next :meth:connect / :meth:stage call sees a fully booted runtime ready to accept a fresh raw-REPL session.

CircuitPython transport

chumicro_deploy.circuitpython_transport

CircuitPython device transport using pyserial raw REPL.

Uses pyserial to connect to CircuitPython boards via serial and execute code through raw REPL mode (Ctrl-A). Two modes:

  • RAM mode (default): All source code is sent inline through the raw REPL, with no file copy or flash writes required.
  • Flash mode: Files are copied to the CIRCUITPY USB drive for persistent deployment. Autoreload is managed via raw REPL commands.

Raw REPL protocol: 1. Ctrl-C × 2 interrupts any running code. 2. Ctrl-A enters raw REPL (prompt: raw REPL; CTRL-B to exit\r\n>). 3. Send code bytes, terminated with Ctrl-D. 4. Response: OK<stdout>\x04<stderr>\x04>.

PostStageStep

Bases: Enum

Which of the two post-rsync steps a deploy context runs: soft- reboot into the freshly-staged code.py, or keep the live raw- REPL session for the harness to drive.

Every context stages through the same clean-slate rsync + :data:flash_drive.DEVICE_KEEP_SET, so the bytes reach the board identically for a project, an example, and a functional test. Only the step taken once the bytes have landed differs, and only two values exist. Naming them here keeps the divergence explicit and gives the planned drift lint a symbol to anchor on instead of two unrelated methods that happen to differ.

  • :attr:SOFT_REBOOT_AND_TAIL: Ctrl-D soft-reboot so the board runs the freshly-staged code.py, then capture its serial output.
  • :attr:HARNESS_EXEC_OVER_REPL: keep the live raw-REPL session alive (a soft-reboot would tear down state the harness drives over that session), refresh the FAT cache with a cheap directory walk, and let the caller exec the harness and collect asserts.

The chosen value never changes how the bytes got there: everything up to and including the rsync runs the same code path.

SerialPort

Bases: Protocol

Structural interface for a serial port.

Implementations can satisfy this protocol without importing pyserial.

CircuitpythonTransportError

Bases: DeviceTransportError

Raised when a CircuitPython serial operation fails.

CircuitpythonMidDeployDisconnected

Bases: CircuitpythonTransportError, MidDeployDisconnected

Raised when the device drops mid-deploy.

The CircuitPython face of :class:.protocol.MidDeployDisconnected (which owns the message shape and the :attr:cause attribute). The wrapped :class:OSError is typically a :class:serial.SerialException here.

CircuitpythonTransport

Transport for CircuitPython boards via pyserial raw REPL.

Supports two modes:

  • ram (default): all source code is sent inline through the raw REPL, with no file copy or mounting needed.
  • flash: files are copied to the CIRCUITPY USB drive for persistent deployment.

Parameters:

Name Type Description Default
address str

Serial port path (e.g. /dev/cu.usbmodem14101).

required
baudrate int

Serial baud rate. Defaults to 115200.

115200
timeout float

Read timeout in seconds.

DEFAULT_TIMEOUT
mode str

"ram" (default) or "flash".

'ram'
serial_port_factory Callable[..., object] | None

Callable that creates a serial port object. Accepts (port, baudrate, timeout) keyword arguments. Defaults to serial.Serial. Inject a fake for testing.

None
time TimeSource | None

Object providing monotonic() and sleep() methods. Defaults to Python's time module. Inject FakeTime for deterministic tests with no wall-clock waits.

None

The CIRCUITPY drive (mode="flash") is auto-resolved at deploy time via :func:_circuitpy_volume_candidates and verified against the connected board's identity by :meth:_verify_drive_for_board. No per-transport drive path is stored. Resolving by board UID at deploy time lets a host with two boards pick the right drive even when the OS gives the same mount name to a different board across reboots.

staged_sources property

Return the staged module sources, or None if not staged.

connect()

Open the serial port and enter raw REPL mode.

Sends Ctrl-C × 2 to interrupt any running code, then Ctrl-A to enter raw REPL.

Raises:

Type Description
CircuitpythonTransportError

If the serial port cannot be opened or raw REPL prompt is not received.

stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)

Read source files into memory for inline execution.

Sister of :meth:deploy_files that takes library/test/harness directories (source_dirs + test_files + harness_source) instead of a flat files: dict[device_path, bytes].

In RAM mode, source code is read and stored for embedding into the bootstrap code block sent via raw REPL.

In flash mode, source packages are copied to the CIRCUITPY USB drive after disabling autoreload.

Parameters:

Name Type Description Default
source_dirs list[Path]

Library src/ directories to include.

required
test_files list[Path]

Test files to stage (stored for bootstrap generation).

required
harness_source Path

Path to the test harness src/ directory.

required
extra_modules list[Path] | None

Optional sibling Python files to register as importable on the device alongside library sources. In RAM mode they join staged_sources; in flash mode they land at the drive root next to the test files.

None
extra_files dict[str, bytes] | None

Non-Python files to land at named device paths, typically {"/runtime_config.msgpack": <bytes>} so test code can call chumicro_config.load_runtime_config(). RAM mode raises :class:UnsupportedExtraFilesError because there's no writable device-side filesystem to land bytes on; flash mode writes each file to the CIRCUITPY drive alongside library sources.

None

Raises:

Type Description
UnsupportedExtraFilesError

mode == "ram" and extra_files is non-empty. Switch the device's deploy_mode to "flash" to fix.

execute(bootstrap_script, *, on_line=None)

Send a code block through raw REPL and return captured stdout.

Parameters:

Name Type Description Default
bootstrap_script str

Python code to execute on the device.

required
on_line Callable[[str], None] | None

Optional callback invoked with each stdout line as it arrives over the serial link, before this method returns. Lines are delivered without their trailing newline; \r\n is normalized to \n. When None, no streaming dispatch happens and the captured stdout is still returned in the usual request/response shape.

None

Returns:

Type Description
str

Captured stdout from the device.

Raises:

Type Description
CircuitpythonTransportError

If stage() has not been called, the device returns an error, or communication fails.

execute_scripts(bootstrap_scripts, *, on_line=None)

Execute multiple raw-REPL scripts in one interpreter session.

Large CircuitPython RAM-mode payloads are more reliable when split into smaller scripts. Each script leaves the interpreter in raw REPL ready for the next chunk.

Parameters:

Name Type Description Default
bootstrap_scripts list[str]

Ordered raw-REPL scripts to execute.

required
on_line Callable[[str], None] | None

Optional callback threaded through each per-script :meth:execute call. Lines from all scripts are delivered in arrival order under a single callback.

None

Returns:

Type Description
str

Stdout from the final script.

reset_into_bootloader()

Reset into the UF2 bootloader via microcontroller module.

microcontroller.on_next_reset(RunMode.BOOTLOADER) + microcontroller.reset() is CircuitPython's documented way to enter bootloader mode. CP implements it across all ESP32-S2/S3/C6/P4 + RP2040/RP2350 + most SAMD/nRF52 ports, but whether the call actually puts a particular board in bootloader mode depends on that board's HAL build. Treat success as observable, not declared: the raw-REPL link drops as the chip resets (expected), and a new bootloader port / UF2 drive becomes visible to whatever polls for one. Don't add board-specific branching here.

Closes the serial port directly so a subsequent :meth:disconnect becomes a no-op. The USB link is gone on purpose and running the normal restore dance (_enter_raw_repl + autoreload-on + Ctrl-D) against a dying link only produces misleading warnings.

probe_implementation()

Query sys.implementation on the board for PR-summary metadata.

Uses the persistent raw REPL _send_repl_command helper so no staging is required. sys.implementation is a built-in. Failures are swallowed (None returned) so a flaky firmware never blocks the real test run.

Returns:

Type Description
DeviceImplementation | None

class:DeviceImplementation on success, or None if

DeviceImplementation | None

the probe could not complete.

probe_free_memory()

Return free heap bytes reported by the connected board.

inline_script_budget_bytes()

Return a conservative raw-REPL script budget based on live heap.

run_script(script, *, timeout=10.0)

Run script via the persistent raw REPL with no staging.

Bypasses the stage()-first precondition that :meth:execute enforces. script must be self-contained (no imports of staged modules).

Parameters:

Name Type Description Default
script str

Self-contained Python source to run.

required
timeout float

Reserved for protocol parity; the underlying _send_repl_command uses its own deadline strategy.

10.0

Returns:

Type Description
str

Captured stdout from the device.

soft_reset()

Soft-reset the interpreter and re-enter raw REPL.

Flushes any host-cached FAT writes to the CIRCUITPY volume (flash mode only), exits raw REPL (Ctrl-B), sends Ctrl-D to trigger a soft reboot (which clears sys.modules and all interpreter state), waits for the reboot to complete, then re-enters raw REPL.

Leaves the interpreter clean, with previous modules evicted from RAM.

Raises:

Type Description
CircuitpythonTransportError

If raw REPL cannot be re-established after the reset.

recover()

Attempt to recover raw REPL after a failed test.

Sends Ctrl-C to interrupt any running code, drains stale output, then re-enters raw REPL mode. Call this after a test error before running the next test.

Raises:

Type Description
CircuitpythonTransportError

If raw REPL cannot be re-established.

deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, tail_seconds=None, clean=False)

Deploy files and execute entrypoint in the configured mode.

In flash mode, every entry of files is written to the CIRCUITPY USB drive (auto-detecting the mount when not configured), the volume is flushed, and the entrypoint runs through the persistent raw REPL. Autoreload is disabled during writes so the board does not reset mid-deploy.

In RAM mode, no filesystem is touched. Every non-entrypoint .py file is injected into sys.modules via the class-as-module pattern (see :func:build_circuitpython_deploy_scripts) and the entrypoint runs as __main__. Non-.py payload is silently skipped.

Parameters:

Name Type Description Default
files dict[str, bytes]

On-device-path -> bytes mapping. In flash mode the leading slash is stripped before joining with the drive mount. In RAM mode the path derives the dotted module name (/lib/foo/bar.py -> foo.bar).

required
entrypoint str

On-device path, must be a key of files.

required
on_file_staged Callable[[str], None] | None

Per-file callback after each write (flash mode) or before the inline scripts run (RAM mode, in sorted-key order).

None
on_execute_line Callable[[str], None] | None

Callback invoked once per captured output line, in order, after the entrypoint returns.

None
tail_seconds float | None

Flash mode only. Override for how long serial output is captured after the soft-reboot. None uses :attr:timeout, 0.0 exits immediately without capturing. RAM mode ignores this.

None
clean bool

Flash mode only. When True, rsync --delete prunes drive files outside the payload.

False

Returns:

Type Description
str

Combined stdout from the entrypoint execution.

Raises:

Type Description
CircuitpythonTransportError

Port not connected, entrypoint missing from files, or (flash mode) the CIRCUITPY drive cannot be located.

list_files_in_scope(*, clean_slate=False)

Implements the contract on :meth:TransportProtocol.list_files_in_scope.

Flash mode walks the CIRCUITPY USB drive directly via stdlib pathlib, which is faster and simpler than a raw-REPL round-trip, and the drive's contents are the device's filesystem. Pairs :meth:_resolve_circuitpy_drive with :meth:_verify_drive_for_board so the macOS multi-board mount-order swap is silently corrected to the connected board's actual mount before the walk. RAM mode returns an empty list.

delete_files(paths)

Delete paths from the CIRCUITPY drive.

Flash mode only. RAM mode is a no-op since nothing was ever written to flash. Each path is normalized to a leading-slash form, joined under the CIRCUITPY mount point, and unlinked best-effort. Missing paths and per-path errors are tolerated silently so a transient I/O hiccup never blocks the deploy that follows.

Pairs _resolve_circuitpy_drive with _verify_drive_for_board so the macOS first-mount-wins behavior cannot unlink the other board's files on multi-board hosts.

Uses :meth:pathlib.Path.unlink rather than rsync --delete on purpose: rsync's delete semantics are "remove anything in DEST not in SRC", which is the wrong shape for "delete these specific files." Unlink also dodges FAT32's data-write reliability concerns by only touching directory entries, no payload bytes. The stale set is recomputed on every deploy, so a swallowed error here just retries next time.

clear_entrypoints()

Unlink code.py / main.py and confirm they are gone.

Flash mode only. RAM mode never writes a persistent entrypoint, so there is nothing to clear. Host-side pathlib.unlink (a directory-entry-only op with no FAT data write), then flush_volume so the deletion is on the physical medium before a soft-reboot re-reads FAT, then a bounded poll that the drive no longer shows them. Raises if an entrypoint is still present after the flush + poll budget, because a surviving code.py is exactly the soft-reboot race this removes.

wipe_filesystem()

Implements the contract on :meth:TransportProtocol.wipe_filesystem.

Drives the on-board nuclear option through raw REPL: the call reformats the FAT volume and reboots the board. The host-side serial session goes away mid-call as USB-CDC drops, and the failure that surfaces is expected and swallowed. After waiting for the reformat + reboot to settle the port is re-opened and raw REPL re-entered, leaving the transport in the same state :meth:connect does so a follow-up :meth:deploy_files works without further setup.

disconnect()

Close the serial port and clear staged data.

Pure teardown. Sends a bare Ctrl-B (exits raw REPL when in raw REPL, a no-op control byte otherwise) so the next serial consumer never finds the board parked in raw REPL, then closes the port. No Ctrl-C, because that would interrupt any code.py left running on the board. No supervisor.runtime.autoreload = True, because flipping autoreload back on outside an active raw-REPL session can layer an autoreload-driven soft-reboot on top of one already in flight, which can wedge ESP32-S2 USB-CDC firmware.

Tolerates an already-closed port (e.g. after :meth:reset_into_bootloader nulled :attr:_port): the method finds nothing to close and only clears :attr:_staged_sources.

CircuitPython bootstrap helpers

chumicro_deploy.circuitpython_bootstrap

CircuitPython bootstrap code generator.

Generates inline code blocks that can be sent through the CircuitPython raw REPL. Two builders share the same helper-function prefix:

  • :func:build_circuitpython_bootstrap_scripts is the test-harness flavor. Registers library modules via the class-as-module pattern (no types.ModuleType on CircuitPython), inlines a test file, and runs it through chumicro_test_harness.runner.run_module.
  • :func:build_circuitpython_deploy_scripts is the deploy flavor. Registers every non-entrypoint file as an importable module, then exec()-s the entrypoint as __main__. No test-harness dependency is needed, and the deploy tail is two lines long, kept as an inline string constant rather than a separate template file.

Both share the static helper functions in circuitpython_bootstrap_template.txt (read once per builder call). _register_stub and _populate_module are universal. Only the dynamic parts (module sources, entrypoint / test source, filter) are generated here.

CircuitpythonBootstrapTooLargeError

Bases: ValueError

Raised when an inline CircuitPython bootstrap is too large to send safely.

build_circuitpython_bootstrap_scripts(staged_sources, test_file, *, name_filter=None, max_chunk_size_bytes=DEFAULT_INLINE_SCRIPT_BUDGET_BYTES)

Generate chunked raw-REPL scripts for CircuitPython RAM-mode tests.

CircuitPython raw REPL buffers and compiles each submitted script before it executes. Large one-shot payloads can therefore exhaust heap even when the final imported modules would fit. This builder splits the inline bootstrap into smaller scripts that can be executed sequentially in one interpreter session.

Parameters:

Name Type Description Default
staged_sources list[tuple[str, str]]

List of (dotted_module_name, source_text) tuples.

required
test_file Path

Path to the test file to execute.

required
name_filter str | None

Optional substring filter passed to run_module.

None
max_chunk_size_bytes int

Maximum encoded size for any single submitted raw REPL script.

DEFAULT_INLINE_SCRIPT_BUDGET_BYTES

Returns:

Type Description
list[str]

Ordered list of Python source strings to execute sequentially.

build_circuitpython_deploy_scripts(files, entrypoint, *, max_chunk_size_bytes=DEFAULT_INLINE_SCRIPT_BUDGET_BYTES)

Generate raw-REPL scripts for CircuitPython RAM-mode deploy.

The returned scripts inject every non-entrypoint file as an importable module via sys.modules, then exec() the entrypoint as __main__. No chumicro_test_harness dependency is required on the device.

Module naming follows CircuitPython's sys.path convention: files under /lib/ are stripped of that prefix, so /lib/foo.py registers as foo and /lib/foo/bar.py as foo.bar. Files at the device root (/foo.py) register as top-level foo. Non-.py files are silently skipped. RAM-mode deploy has no device filesystem to write them to, so shipping non-Python assets requires flash mode.

Parameters:

Name Type Description Default
files dict[str, bytes]

On-device-path -> bytes mapping.

required
entrypoint str

On-device path exec'd as __main__. Must be a key of files.

required
max_chunk_size_bytes int

Maximum encoded size for any single submitted raw REPL script. Defaults to :data:DEFAULT_INLINE_SCRIPT_BUDGET_BYTES; pass a tighter budget derived from gc.mem_free() when known.

DEFAULT_INLINE_SCRIPT_BUDGET_BYTES

Returns:

Type Description
list[str]

Ordered list of Python source strings to execute sequentially.

Raises:

Type Description
ValueError

entrypoint is missing from files or max_chunk_size_bytes is not positive.

CircuitpythonBootstrapTooLargeError

A single chunk exceeds max_chunk_size_bytes.

Flash drive helpers

chumicro_deploy.flash_drive

CircuitPython flash-mode USB-drive staging and FAT32 hygiene helpers.

Every helper here operates on filesystem paths or subprocess calls and has no transport state to carry, so they live as module-level functions instead of methods on a class. :func:flush_volume takes an injected sleep callable so tests can skip the real settle delay.

FlashDriveError

Bases: Exception

Raised when a flash-drive staging operation fails.

FatVolumeCorruptionError

Bases: FlashDriveError

Raised when a FAT volume carries torn directory entries.

A torn entry is one the host can list via readdir but whose stat fails with EINVAL and which cannot be unlinked, the state a FAT directory is left in when cached host-side metadata writes land across a board reset. The volume cannot be repaired in place; recovery is a full board-filesystem reformat.

Attributes:

Name Type Description
corrupted_paths

Volume-relative paths of the torn entries, as enumerated by :func:scan_for_torn_directory_entries. May be empty when only rsync's stderr identified the corruption.

compute_rsync_timeout_seconds(staging_size_bytes)

Pick an rsync timeout proportional to staging-tree size.

Formula::

max(RSYNC_TIMEOUT_BASE_SECONDS + size_mb * RSYNC_TIMEOUT_PER_MB_SECONDS,
    RSYNC_TIMEOUT_MIN_SECONDS)

Scales the deadline so fast boards fail fast on a real wedge while slow boards still have headroom to finish.

Parameters:

Name Type Description Default
staging_size_bytes int

Sum of file sizes in the local staging tree, typically from :func:_directory_size_bytes.

required

merge_packages(source_directory, staging_destination, *, target_runtime=None, include_test_support=False)

Copy top-level packages from a source directory to a staging dir.

Merges into the staging destination using dirs_exist_ok=True so multiple source directories can contribute packages. Operates on the local filesystem (not the USB drive), so shutil.copytree is reliable here.

Parameters:

Name Type Description Default
source_directory Path

A src/ directory containing packages.

required
staging_destination Path

Local staging directory to merge into.

required
target_runtime str | None

When set, .py files carrying a __chumicro_runtimes__ marker for a different runtime are skipped (in addition to __pycache__ / *.pyc). None (the default) keeps every .py file regardless of its marker.

None
include_test_support bool

When False (the default), .py files marked with __chumicro_test_support__ = True are skipped. Pass True to merge them too, e.g. when staging a test-fake bundle for on-device test runs.

False

rsync(source, destination, *, delete=True, additional_excludes=(), timeout=None)

Rsync a source directory's contents to a destination.

Single FAT-write primitive: --checksum verifies content (FAT32 timestamps are unreliable) and --inplace avoids temp-file rename races on FAT32. Two parameter shapes:

  • Clean push: delete=True with additional_excludes=DEVICE_KEEP_SET. Clean slate, only the closed keep set survives.
  • Additive push: delete=False with no extra excludes. Stale files persist. Used when other board files are hand-managed.

The base exclude set (build artifacts + macOS noise / sentinel dirs) is shared and unconditional; see :data:_BASE_RSYNC_EXCLUDES.

Parameters:

Name Type Description Default
source Path

Source directory whose contents to sync.

required
destination Path

Destination directory.

required
delete bool

When True, pass --delete so files in destination but not source are removed (clean slate; only additional_excludes survive). False is the additive shape, where stale files persist.

True
additional_excludes tuple[str, ...] | list[str]

Extra basenames to add to --exclude. Pass :data:DEVICE_KEEP_SET on clean pushes so --delete doesn't wipe the device-required keep set; leave empty for additive pushes.

()
timeout float | None

Override the auto-computed timeout (seconds). Default None lets :func:compute_rsync_timeout_seconds pick a value scaled to the staging-tree size. Pass an explicit value when a deterministic deadline is needed.

None

Raises:

Type Description
FlashDriveError

If rsync is not installed or the sync fails.

verify_rsync(source, destination, *, additional_excludes=(), timeout=30.0)

Confirm destination's contents match source via rsync dry-run.

Runs rsync --recursive --checksum --dry-run --itemize-changes (the same flags :func:rsync uses, plus dry-run + itemize) and returns the list of paths rsync reports as needing update. When the previous real :func:rsync call committed every byte, the list is empty. Non-empty means content on the volume diverged from the staging tree, which is the signal for FAT corruption, USB-MSC partial-write, or a drive that quietly went read-only after the writes started.

Itemize-changes flag positions: position 1 is the update marker (< / > / c / h mean "would transfer"; . / * mean "no update needed"). We filter on the first character so cosmetic time / permission deltas (.f..T....) don't fire a false positive. Only real content / size diffs do.

Parameters:

Name Type Description Default
source Path

Staging tree that was rsynced to destination.

required
destination Path

Mount point that should now mirror source.

required
additional_excludes tuple[str, ...] | list[str]

Extra basenames passed through to --exclude so the dry-run scope matches the original rsync's scope exactly.

()
timeout float

Subprocess deadline (seconds). Verification reads every file from the FAT volume.

30.0

Returns:

Type Description
list[str]

Sorted list of source-relative paths that the verification

list[str]

rsync would update. Empty on success.

Raises:

Type Description
FlashDriveError

rsync is missing or the subprocess errored for a reason other than "would-update detected" (which shows up in stdout, not as a non-zero exit).

format_fat_corruption_error(drive_path, corrupted_paths, *, detail='')

Build the single actionable message for a torn-FAT volume.

Names every corrupted path plus the one recovery command that works, so a sweep fails once with instructions instead of every test failing with repeated rsync stderr noise. The lead phrase FAT directory entries is load-bearing: the recovery classifier's FAT_VOLUME_CORRUPT pattern matches on it, so reword only together with the matching pattern in recovery.py.

Parameters:

Name Type Description Default
drive_path Path

Mount point of the corrupted volume.

required
corrupted_paths Iterable[str]

Volume-relative paths of the torn entries. May be empty when only rsync stderr identified the state.

required
detail str

Optional raw diagnostic (typically rsync stderr) to embed for the report.

''

scan_for_torn_directory_entries(drive_path, *, scandir=os.scandir)

Stat every entry under drive_path and collect EINVAL casualties.

Walks the volume with scandir and calls stat on each entry. A healthy entry stats cleanly; a torn FAT directory entry is listed by readdir but fails stat with EINVAL, so any EINVAL here is the corruption signature. Directories whose own listing fails with EINVAL are recorded the same way. Other per-entry OSError values (a file deleted mid-walk, a kernel-locked macOS noise directory) are skipped: they are not the torn-entry state this scan exists to find.

Parameters:

Name Type Description Default
drive_path Path

Mount point to walk.

required
scandir Callable[[Path], Iterable]

Directory-listing callable, os.scandir by default. Injectable so tests can model the torn-entry behavior (readdir succeeds, stat raises EINVAL) without a real corrupted FAT volume.

scandir

Returns:

Type Description
list[str]

Sorted volume-relative paths of every torn entry. Empty on a

list[str]

healthy volume.

strip_extended_attributes(path)

Remove macOS extended attributes from all files under path.

Extended attributes (xattrs) cause slow transfers to FAT32 volumes and generate ._ resource fork files. Stripping them from the staging directory before rsync prevents these artifacts from reaching the device.

No-op on non-macOS platforms.

Parameters:

Name Type Description Default
path Path

Root directory to strip recursively.

required

clean_dot_files(drive_path)

Merge or remove ._ resource fork files on a FAT32 volume.

macOS creates ._ files on FAT32 drives even when rsync excludes them, because the OS itself writes them during filesystem operations. dot_clean merges these back into the native file or removes them if the native file is absent.

No-op on non-macOS platforms.

Parameters:

Name Type Description Default
drive_path Path

Mount point of the FAT32 volume.

required

plant_macos_sentinels_in_staging(staging_path)

Plant macOS skip-sentinels into a local staging directory.

The sentinels (see :data:_MACOS_SKIP_SENTINELS for what each suppresses) go at the staging-tree root so rsync ships them onto CIRCUITPY in the same pass as the payload. No host-side write to the live drive before rsync starts, since every such write can wedge rsync in uninterruptible kernel I/O.

No-op on non-macOS platforms.

Parameters:

Name Type Description Default
staging_path Path

Local staging-tree root. rsync will copy everything under this into the CIRCUITPY drive root.

required

cleanup_macos_noise_dirs_post_rsync(drive_path)

Remove already-accumulated macOS noise directories from the drive.

Called after rsync so the wedge-risky on-drive writes stay inside the single rsync pass. Clears the noise dirs a drive picked up from earlier macOS mounts (:data:_MACOS_NOISE_DIRS); rsync --delete can't, since they're excluded and partly kernel-locked on FAT. shutil.rmtree(ignore_errors=True) walks them best-effort, so a drive the sentinels already keep clean is a no-op.

No-op on non-macOS platforms.

Parameters:

Name Type Description Default
drive_path Path

Mount point of the CIRCUITPY drive.

required

flush_volume(drive_path, *, sleep, settle_delay=FLUSH_SETTLE_DELAY)

Flush pending writes to the volume containing drive_path.

Three layers, weakest to strongest: the sync command (or os.sync() off macOS) schedules every volume's dirty buffers; F_FULLFSYNC on the mount point (via :func:_force_full_flush_to_media) waits until this volume's writes have actually reached the medium; the settle delay gives the board's USB-MSC controller time to finish its own FAT bookkeeping. All three must have completed before anything resets the board, because a reset that lands while host-side FAT metadata is still cached can tear directory entries into an unreadable, undeletable state.

The settle delay goes through the injected sleep callable so tests can use a fake time source to skip it without sleeping for real.

Parameters:

Name Type Description Default
drive_path Path

Path on the volume to flush.

required
sleep Callable[[float], None]

Callable that sleeps for the given number of seconds.

required
settle_delay float

Seconds to wait after the sync.

FLUSH_SETTLE_DELAY

Test fakes

chumicro_deploy.testing

Test fakes for the device transport layer.

Exports :class:FakeTransport, :class:FakeSerialPort, :class:FakeTime, and :func:isolate_from_host_filesystem. See each one's docstring for shape and usage. Typical injection::

from chumicro_deploy.testing import FakeSerialPort, FakeTime

transport = CircuitpythonTransport(
    address="/dev/cu.fake",
    port_factory=lambda *_args, **_kwargs: FakeSerialPort(...),
    time=FakeTime(),
)

FakeTime

Deterministic seconds-domain time source for host-side tests.

Bundles monotonic() and sleep() into a single injectable object that satisfies :class:chumicro_deploy.protocol.TimeSource (both device transports take one). monotonic() is stable: repeated calls return the same value until the clock is explicitly advanced. sleep(duration) auto-advances the clock by duration without any real wait, so production code that sleeps still moves the fake clock forward. advance(seconds) moves the clock forward explicitly, for scenarios where production does not sleep but the test needs to simulate elapsed time (e.g., timeout expiry).

Example::

fake = FakeTime()
assert fake.monotonic() == 0.0

fake.sleep(1.5)
assert fake.monotonic() == 1.5

fake.advance(0.5)
assert fake.monotonic() == 2.0

__init__(start=0.0)

Create a fake time source starting at start seconds.

Parameters:

Name Type Description Default
start float

Initial monotonic value in seconds.

0.0

monotonic()

Return the current fake time in seconds.

sleep(duration)

Advance the clock by duration seconds (no wall-clock wait).

advance(seconds)

Move the clock forward by seconds.

FakeSerialPort

Simulates a pyserial Serial port for transport testing.

Records all writes and returns canned responses for reads. Each entry in read_responses is either bytes (returned verbatim on the matching read() call) or an instance of BaseException (raised on that call). The exception form scripts "first read returns bytes, second read drops the cable" scenarios without subclassing the fake.

Instances are callable and return themselves, so a FakeSerialPort instance can be passed directly as a transport's serial_port_factory kwarg with no wrapper closure needed. Pass open_error to script "factory raises when the transport asks for a port" without writing a custom closure.

in_waiting property

Return how many bytes are available to read.

Scripted BaseException entries are reported as having in_waiting == 1 so polling loops actually call :meth:read (which then raises) instead of looping past the scripted disconnect.

__call__(*_args, **_kwargs)

Return this port, or raise the scripted open_error.

Lets the instance double as a serial_port_factory callable. Accepts both positional and keyword args (different transports call factories with different signatures); all are ignored. Use an explicit closure when factory-arg assertions are needed.

read(size=1)

Return the next canned response, or raise the scripted exception.

write(data)

Record a write, or raise the configured exception.

close()

Mark the port as closed.

reset_input_buffer()

No-op for fake.

FakeTransport dataclass

In-memory fake that records transport calls and returns canned output.

Satisfies both :class:TransportProtocol and :class:ExtendedTransportProtocol, so the same fake stands in for either a MicroPython or CircuitPython transport. The chunked-execute helpers fall back to :meth:execute when no chunked-specific behavior is configured.

connect()

Record a connect call; raise :attr:connect_raises if set.

stage(source_dirs, test_files, harness_source, *, extra_modules=None, extra_files=None, include_test_support=False)

Record a stage call.

Parameters:

Name Type Description Default
source_dirs list[Path]

Library source directories.

required
test_files list[Path]

Test file paths.

required
harness_source Path

Harness source directory.

required
extra_modules list[Path] | None

Sibling Python files to register as importable on the device.

None
extra_files dict[str, bytes] | None

Non-Python files to land at named device paths (typically {"/runtime_config.msgpack": <bytes>}). A non-empty dict in RAM mode raises :class:UnsupportedExtraFilesError.

None

Raises:

Type Description
UnsupportedExtraFilesError

mode == "ram" and extra_files is non-empty. RAM mode bypasses the device filesystem entirely, so there's nowhere to land bytes. Switch the device's deploy_mode to "flash" before calling.

execute(bootstrap_script, *, on_line=None)

Record an execute call and return canned output.

Returns the head of :attr:outputs when non-empty (popped), otherwise :attr:execute_output. Raises :attr:execute_raises after recording when set.

When on_line is provided, dispatches one call per line of the returned string (split via :meth:str.splitlines) before returning, the same shape the real transports use, so a test can drive marker-style stdout coordination against the fake without wiring a real serial stream.

run_script(script, *, timeout=10.0)

Record a run_script call and return canned output.

Parameters:

Name Type Description Default
script str

Self-contained Python source to run with no staging.

required
timeout float

Recorded with the call but not enforced, since the fake transport's "device" responds instantly.

10.0

Returns:

Type Description
str

The configured execute_output (run_script shares

str

the same canned-response slot; tests that need separate

str

run_script / execute outputs can override after

str

construction).

execute_scripts(bootstrap_scripts, *, on_line=None)

Record a chunked-execute call and return the configured output.

Raises :attr:execute_raises after recording when set.

When :attr:outputs is non-empty, pops a single head value and returns it, treating the whole chunked execute as one batched operation, with no synthetic per-script execute entries. Otherwise records synthetic per-script execute entries (one per chunk) and returns :attr:execute_output.

on_line mirrors the real-transport pass-through: the batched head dispatches one call per line; the per-script branch threads the callback into each :meth:execute call.

probe_free_memory()

Record a probe call and return the configured free-heap value.

inline_script_budget_bytes()

Return half the configured free-memory budget.

probe_implementation()

Record a probe call and return the canned result.

soft_reset()

Record a soft_reset call.

recover()

Record a recover call; raise :attr:recover_raises if set.

disconnect()

Record a disconnect call.

reset_into_bootloader()

Record the call and return the configured result.

Defaults to True (pretending the dispatch succeeded) for the common branch; override via :attr:bootloader_reset_result to simulate the "runtime doesn't support bootloader entry" fallback.

deploy_files(files, entrypoint, *, on_file_staged=None, on_execute_line=None, follow='exec', clean=False, tail_seconds=None)

Record a deploy_files call and return the configured output.

Emits on_file_staged per file (sorted for deterministic order) and on_execute_line per line of execute_output before returning. The calls entry uses a dict-of-bytes + entrypoint + follow tuple so the payload, callback ordering, and follow mode are all observable.

The follow kwarg accepts the same values as :class:MicropythonTransport.deploy_files ("exec" or "soft_reboot"). The fake records it but doesn't otherwise change behavior.

tail_seconds is recorded on :attr:last_tail_seconds so tests can assert the capture-window override.

list_files_in_scope(*, clean_slate=False)

Return on-device paths in scope, drawn from :attr:device_files.

clean_slate=True widens to every device file except the closed keep set (:data:flash_drive.DEVICE_KEEP_SET). False keeps the :func:is_in_deploy_scope filter so the additive path leaves out-of-scope files alone.

delete_files(paths)

Remove paths from the simulated on-device state.

clear_entrypoints()

Drop the simulated code.py / main.py entrypoints.

wipe_filesystem()

Erase every simulated on-device file.

In flash/copy mode the whole user filesystem (in-scope + out- of-scope alike) is gone after this returns. In RAM/mount mode the call is a no-op, so both branches exercise against the same fake.

isolate_from_host_filesystem(monkeypatch, *, circuitpy_drives=None)

Sever flash_drive's links to the real macOS host filesystem.

Two cleanups in one, both load-bearing for hermetic tests:

  1. macOS shell helpers → instant fakes. flash_drive calls subprocess.run(["sync"]) / ["xattr", ...] / ["dot_clean", ...] / ["mdutil", ...] on macOS. Each blocks for seconds on a busy host (concurrent test workers, IDE indexing, git activity). Real rsync is left intact, because tests rely on its file-copy side effect.
  2. CIRCUITPY drive scanning → controllable. Production calls :func:chumicro_deploy.circuitpy_drive._circuitpy_volume_candidates to find /Volumes/CIRCUITPY* mounts. On a dev box with a real board plugged in, an unstubbed test will write+unlink probe files onto the real device's filesystem. Pass circuitpy_drives to point at a test-controlled directory; pass None (default) to make scans return empty so tests don't depend on host state.

Per-test monkeypatches override these. Calling monkeypatch.setattr("chumicro_deploy.circuitpy_drive ._circuitpy_volume_candidates", ...) again later in the same test replaces the stub for that test only.

Typical usage as a module-level autouse fixture::

@pytest.fixture(autouse=True)
def _isolate(monkeypatch):
    isolate_from_host_filesystem(monkeypatch)