Overview

What is FrameX?

FrameX is a lightweight, pluggable Python framework designed for building modular and extensible algorithmic systems.

It provides a clean architecture that supports dynamic plugin registration, isolated execution, and secure invocation, making it well-suited for multi-algorithm collaboration, heterogeneous task scheduling, and distributed deployments.

Each algorithm can be developed, deployed, and loaded as an independent plugin, achieving infinite scalability.

algov2
A Comparison Between the v2 (Regular FastAPI) and v3 (FrameX) Architectures

Note:

  • v2 can be considered a regular FastAPI project.

Key Features

  • Plugin-Based Architecture
    Algorithms are encapsulated as independent plugins, which can be added, removed, or updated without impacting others.
  • Distributed Execution with Ray
    Optional Ray integration delivers high concurrency, high throughput, and resilience against blocking tasks.
  • Cross-plugin Calls
    Enables interaction between local and remote plugins. If a plugin is not available locally, the system automatically routes the request to the corresponding cloud plugin.
  • Backward Compatibility
    FrameX can seamlessly forward requests to standard FastAPI endpoints, enabling smooth integration without code changes.
  • Streaming Support
    Native support for streaming responses, suitable for long-running or large-scale inference tasks.
  • Built-in Observability
    Integrated logging, tracing, and performance monitoring to ease debugging and root-cause analysis.
  • Flexible Configuration & Tooling
    Clean configuration management (.toml, .env) plus scaffolding, packaging, and CI/CD integration for automation.
algov2
FrameX hub

Application Scenarios

  • Quick Onboarding & Project Setup
    New developers can rapidly bootstrap projects and reuse existing algorithms via remote calls, without accessing legacy code.

  • Multi-Team Parallel Development & Isolation
    Different teams manage their own isolated plugin spaces. Access control ensure security and reduce interference.

  • Hybrid Deployment & Smooth Migration
    Supports hybrid calls with other FastAPI services, dynamic endpoint registration, and multi-instance FrameX deployment with inter-instance communication.

  • Modular Delivery & Commercial Licensing
    Deliver selected algorithm modules locally to clients while keeping others as remotely callable services. This supports licensing, pay-per-use, and flexible business models.

Prerequisites

  • Python 3.11+

Quick Demo

Create foo.py file

from typing import Any
from pydantic import BaseModel

from framex.consts import VERSION
from framex.plugin import BasePlugin, PluginMetadata, on_register, on_request


__plugin_meta__ = PluginMetadata(
    name="foo",
    version=VERSION,
    description="A simple Foo plugin example",
    author="touale",
    url="https://github.com/touale/FrameX-kit",
)


class FooModel(BaseModel):
    text: str = "Hello Foo"


@on_register()
class FooPlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)

    @on_request("/foo", methods=["GET"])
    async def foo(self, message: str) -> str:
        return f"Foo says: {message}"

    @on_request("/foo_model", methods=["POST"])
    async def foo_model(self, model: FooModel) -> str:
        return f"Foo received model: {model.text}"#

Run the following command to start the project creation process:

$ PYTHONPATH=. framex run --load-plugins foo
πŸš€ Starting FrameX with configuration:
{
  "host": "127.0.0.1",
  "port": 8080,
  "dashboard_host": "127.0.0.1",
  "dashboard_port": 8260,
  "use_ray": false,
  "enable_proxy": false,
  "num_cpus": 8,
  "excluded_log_paths": []
}
11-05 16:01:13 [SUCCESS] framex.plugin.manage | Succeeded to load plugin "foo" from foo
11-05 16:01:13 [INFO] framex | Start initializing all DeploymentHandle...
11-05 16:01:13 [SUCCESS] framex.plugin.manage | Found plugin HTTP API "['/api/v1/foo', '/api/v1/foo_model']" from plugin(foo)
11-05 16:01:13 [SUCCESS] framex.driver.ingress | Succeeded to register api(['GET']): /api/v1/foo from foo.FooPlugin
11-05 16:01:13 [SUCCESS] framex.driver.ingress | Succeeded to register api(['POST']): /api/v1/foo_model from foo.FooPlugin
INFO:     Started server process [59373]
INFO:     Waiting for application startup.
11-05 16:01:13 [INFO] framex.driver.application | Starting FastAPI application...
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8080 (Press CTRL+C to quit)

Project demo

Install cookiecutter

Make sure you have Python 3.11 or above installed, then execute the following command in the command line:

pip install cookiecutter

Cookiecutter is a CLI tool (Command Line Interface) to create an application boilerplate from a template. It uses a templating system β€” Jinja2 β€” to replace or customize folder and file names, as well as file content. it can help you quickly create a plugin.

Run the following command to start the project creation process:

cookiecutter https://github.com/yourusername/framex-plugin-project-template.git

Project Type

[1/17] Select type
  1 - project
  2 - plugin
  Choose from [1/2] (1):

Select 2 for plugin and press Enter to continue.

Group Name

[2/17] group_name:

This will default based on your selected type.
You can usually just press Enter to accept the default.

Project Name

[3/17] project_name (Demo Project):

Here you enter the name of your project.
For example: demo_project.

Repository Name

[4/17] repo_name (demo_project):

This defaults to the same value as your project name.
Press Enter to accept unless you want a different repo name.

CI Tag

[5/17] ci_tag (k8s_runner_persionnel_matching):

Generated automatically. Leave as default by pressing Enter.

Project URL

[6/17] project_url ():

Generated automatically. Press Enter to accept.

Author

[7/17] author (touale):

Enter the author name.
For example: zhangsan.

Email

[8/17] email (zhangsan@example.com):

Enter the author’s email.
For example: zhangsan@local.com.

Short Description

[9/17] short_description (Behold My Awesome Project!):

Press Enter to keep the default or type your own description.

Version

[10/17] version (0.0.0):

Default is fine, press Enter.

Python Version

[11/17] python_version (3.11):

Press Enter to use 3.11.

Nexus & Other Sources

For the following prompts, press Enter to use defaults unless you need custom values:

[12/17] nexus_source (https://pypi.org): 
[13/17] primary_pip_source (https://pypi.org/simple): 
[14/17] release_pip_source (https://upload.pypi.org/legacy/): 
[15/17] apt_source (https://mirrors.aliyun.com/): 
[16/17] dockerhub_url (docker.io): 
[17/17] build_image: docker.io/yourusername/demoproject

After completing the above operations, you will get the complete project structure:

$ tree demo_project 
demo_project
|-- Dockerfile
|-- LICENSE
|-- README.md
|-- data
|   `-- demo@f738
|-- mypy.ini
|-- poe_tasks.toml
|-- pyproject.toml
|-- releaserc.toml
|-- ruff.toml
|-- src
|   `-- demo_project
|       |-- __init__.py
|       |-- __main__.py
|       |-- consts.py
|       |-- log.py
|       `-- plugins
|           |-- __init__.py
|           `-- demo.py
|-- tests
|   |-- __init__.py
|   `-- test_add.py
`-- uv.lock

8 directories, 21 files

Initialize the plugin

After the plugin is created, the initialization template will automatically create a plugin example for you and open interfaces such as /api/v1/demo_get, /api/v1/demo_post, /api/v1/demo_stream.

You can install dependencies using the following command in the project directory. Note that you need to be in the touale intranet:

uv sync --dev

Run the plugin

Use the following command to run the plugin:

poe server

Note that this will start the interface opened by the plugin under src/plugins, and you will see the log as follows:

$ poe server
Poe => demo_project
09-03 19:20:02 [SUCCESS] framex.plugin.manage | Succeeded to load plugin "demo" from demo_project.plugins.demo
09-03 19:20:02 [INFO] framex | Start initializing all DeploymentHandle...
09-03 19:20:02 [SUCCESS] framex.plugin.manage | Found plugin HTTP API "/api/v1/demo_get" from plugin(demo)
09-03 19:20:02 [SUCCESS] framex.plugin.manage | Found plugin HTTP API "/api/v1/demo_post" from plugin(demo)
09-03 19:20:02 [SUCCESS] framex.plugin.manage | Found plugin HTTP API "/api/v1/demo_stream" from plugin(demo)
09-03 19:20:02 [SUCCESS] framex.driver.ingress | Succeeded to register api(['GET']): /api/v1/demo_get from demo.DemoPlugin
09-03 19:20:02 [SUCCESS] framex.driver.ingress | Succeeded to register api(['POST']): /api/v1/demo_post from demo.DemoPlugin
09-03 19:20:02 [SUCCESS] framex.driver.ingress | Succeeded to register api(['GET']): /api/v1/demo_stream from demo.DemoPlugin
INFO:     Started server process [10859]
INFO:     Waiting for application startup.
09-03 19:20:02 [INFO] framex.driver.application | Starting FastAPI application...
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8080 (Press CTRL+C to quit)

Visit http://127.0.0.1:8080/docs in your browser and you will see the online documentation after the framework loads your plugin.

demo

Basic Usage Overview

This section introduces the basic concepts and workflow for using FrameX as a plugin-based algorithm framework.
It is designed for developers who are starting with FrameX and want to quickly understand how to:

  • Organize their project
  • Register plugins and expose APIs
  • Configure plugins
  • Load and run plugins
  • Debug and test their implementations

1) What you will learn

After reading this section, you will be able to:

  1. Understand the project structure
    Learn how to organize your project files and keep plugins under the plugins/ directory.

  2. Register and expose plugin APIs
    Use __plugin_meta__ = PluginMetadata(...) and @on_request(...) to define plugin metadata and expose endpoints.

  3. Cross-plugin communication
    Call APIs from other plugins using _call_remote_api(...), and handle synchronous, streaming, and function-style calls.

  4. Manage plugin configuration
    Define plugin-specific configuration with pydantic.BaseModel and inject it via config_class.

  5. Load and start plugins
    Load plugins via configuration (config.toml) or dynamically from code, and run the FrameX runtime with framex.run().

  6. Debug and test plugins
    Use FrameX in non-Ray mode for debugging, and leverage FastAPI’s TestClient for writing automated tests.

2) Roadmap of this Section

πŸ‘‰ With these basics, you will be ready to build modular, extensible, and testable algorithmic systems with FrameX.

Project Structure

A typical project generated with FrameX follows a standardized structure to ensure consistency, maintainability, and easy plugin management.

$ tree demo_project
demo_project
|-- LICENSE
|-- README.md
|-- data
|   `-- demo@f738
|-- mypy.ini
|-- poe_tasks.toml
|-- pyproject.toml
|-- releaserc.toml
|-- ruff.toml
|-- src
|   `-- demo_project
|       |-- __init__.py
|       |-- __main__.py
|       |-- consts.py
|       |-- log.py
|       `-- plugins
|           |-- __init__.py
|           `-- demo.py
|-- tests
|   |-- __init__.py
|   `-- test_add.py
`-- uv.lock

1) Key Directories and Files

  • src/ – Main source code directory.
    • main.py: Application entry point.
    • consts.py: Constants (e.g., version).
    • log.py: Logging setup.
    • plugins/: All plugins can be placed here.
  • tests/ – Unit and integration tests.
  • pyproject.toml – Project configuration and dependency management.
  • ruff.toml / mypy.ini – Linting and type-checking configuration.
  • data/ – Sample or runtime data.
  • poe_tasks.toml – Task automation configuration.
  • releaserc.toml – Release configuration.
  • init.py - Single plugin can be placed here.

2) Where should the plugin be placed

The difference from Single plugin management is that one is placed in src/demo_project/__init__.py, and Multiple plugin management is in src/demo_project/plugins/!

Single plugin

If there is only one algorithm or only one plugin, you can directly prevent its entry in the package's init.py.

Example:

src/demo_project/
β”œβ”€β”€ __init__.py # defines one plugin

Multiple plugin

All FrameX plugins can live inside the plugins/ directory. You can structure them in two ways:

A. Easy-file plugin

Each .py file inside plugins/ is treated as one plugin. Example:

src/demo_project/plugins/
β”œβ”€β”€ __init__.py
└── demo.py     # defines one plugin

B. Folder-based plugin

A plugin can also be organized as a package (folder). This is recommended for more complex plugins with multiple modules. Example:

$ tree ../src/framex/plugins -I '__pycache__|*.pyc|*.pyo|*.pyd'
../src/framex/plugins
β”œβ”€β”€ __init__.py
β”œβ”€β”€ echo.py          # single-file plugin
└── proxy/           # folder-based plugin
    β”œβ”€β”€ __init__.py
    β”œβ”€β”€ builder.py
    └── config.py

Here, two plugins are defined:

  • echo – a simple single-file plugin (echo.py)
  • proxy – a more complex plugin implemented as a package with multiple modules

3) Guidelines

  • Every plugin must be placed under plugins/.
  • Each plugin should have a clear entry point (init.py or main class).
  • Use single-file plugins for simple functionality.
  • Use folder-based plugins for more complex features requiring multiple modules.

Plugin Register & API Expose

Every plugin must declare metadata and a plugin class. This section explains how to register a plugin, define its metadata, and expose APIs (HTTP, streaming, or function calls).

1) Plugin Metadata (__plugin_meta__)

Each plugin must define __plugin_meta__ = PluginMetadata(...). Fill it carefully and accuratelyβ€”this information is used for discovery, dependency resolution, routing, and UI/display.

Guidelines for fields:

  • name (required): A short, unique, human-readable plugin name.
  • version (required): Follow semantic versioning (e.g., 1.2.0).
  • description (required): What the plugin does and when to use it.
  • author (required): Owner/maintainer identity (team/company).
  • url (required): Project/repo/docs link for the plugin.
  • required_remote_apis: A list of external API paths your plugin depends on (e.g. endpoints provided by other plugins/services). It enables dependency checks and optional preflight validation.

For example:

__plugin_meta__ = PluginMetadata(
    name="echo",
    version=VERSION,
    description="原η₯žδΌšι‡ε€δ½ θ―΄ηš„话",
    author="原η₯ž",
    url="https://github.com/touale/FrameX-kit",
    required_remote_apis=[],
)

2) Plugin Class & Registration

Each plugin must implement a class decorated with @on_register() and inherit from BasePlugin.

For example:

@on_register()
class EchoPlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)
        # Initialize lightweight resources here (e.g., clients, caches)

    async def on_start(self) -> None:
        # Initialize heavy/async resources here (e.g., DB connections, pools)
        # Called by the runtime when the plugin is starting.
        ...

Notes:

  • Use __init__ for fast, synchronous setup. (Initialization of parameter values)
  • Use on_start for asynchronous or heavy initialization (DB, message queues, engines).

3) Exposing APIs with @on_request(...)

Annotate methods on your plugin class with @on_request(...) to expose them as callable endpoints.

Parameters:

  • path:
    • For ApiType.HTTP: the URL path (e.g. "/echo" or "/v1/echo").
    • For ApiType.FUNC: the symbolic name of the function. If omitted, the method name is used.
  • methods: HTTP methods (e.g. ["GET"], ["POST"]). For function calls, leave it as None.
  • call_type: One of ApiType.HTTP or ApiType.FUNC.
  • stream: Whether the endpoint is streaming. If True, the handler should be an async generator or follow the framework’s streaming contract.

ApiType Explained

  • ApiType.HTTP

    • Exposes the plugin method as an HTTP endpoint.
    • Can be consumed by external clients (e.g., web backends, mobile apps, or third-party services).
    • Can also be used for plugin-to-plugin communication.
    • Offers broader accessibility and is automatically documented under /docs (Swagger/OpenAPI).
    • Recommended when you want to share functionality beyond the plugin boundary.

  • ApiType.FUNC

    • Exposes the plugin method as an internal remote function.
    • Only callable from within the FrameX plugin ecosystem.
    • Provides stricter scoping and reduced exposure surface.
    • Recommended for internal-only calls, utility helpers, or private APIs that should not be published externally.

3.1 HTTP example (non-stream)

__plugin_meta__ = PluginMetadata(
    name="echo",
    version=VERSION,
    description="原η₯žδΌšι‡ε€δ½ θ―΄ηš„话",
    author="原η₯ž",
    url="https://github.com/touale/FrameX-kit",
    required_remote_apis=[],
)


class EchoModel(BaseModel):
    id: int
    name: str = "原η₯ž"


@on_register()
class EchoPlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)

    @on_request("/echo", methods=["GET"])
    async def echo(self, message: str) -> str:
        return message

3.2 HTTP example (streaming)

...
@on_register()
class EchoPlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)

    @on_request(path="/echo_stream", methods=["GET"], call_type=ApiType.HTTP, stream=True)
    async def echo_stream(self, message: str):
        # yield chunked events/messages
        for ch in f"Streaming: {message}":
            yield {"type": "chunk", "data": ch}
        yield {"type": "finish"}

3.3 Function call example

...
@on_register()
class EchoPlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)

    @on_request(call_type=ApiType.FUNC)
    async def add(self, a: int, b: int) -> int:
        return a + b

4) End-to-End Example

# src/__init__.py

import asyncio
from collections.abc import AsyncGenerator
from typing import Any

from pydantic import BaseModel

from framex.consts import VERSION
from framex.plugin import BasePlugin, PluginMetadata, on_register, on_request
from framex.plugin.model import ApiType
from framex.utils import StreamEnventType, make_stream_event

__plugin_meta__ = PluginMetadata(
    name="echo",
    version=VERSION,
    description="原η₯žδΌšι‡ε€δ½ θ―΄ηš„话",
    author="原η₯ž",
    url="https://github.com/touale/FrameX-kit",
    required_remote_apis=[],
)


class EchoModel(BaseModel):
    id: int
    name: str = "原η₯ž"


@on_register()
class EchoPlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)

    @on_request("/echo", methods=["GET"])
    async def echo(self, message: str) -> str:
        return message

    @on_request("/echo_model", methods=["POST"])
    async def echo_model(self, message: str, model: EchoModel) -> str:
        return f"{message},{model.model_dump()}"

    @on_request("/api/v1/echo_stream", methods=["GET"], stream=True)
    async def echo_stream(self, message: str) -> AsyncGenerator[str, None]:
        for char in f"原η₯žηœŸε₯½ηŽ©ε‘€, {message}":
            yield make_stream_event(StreamEnventType.MESSAGE_CHUNK, char)
            await asyncio.sleep(0.1)
        yield make_stream_event(StreamEnventType.FINISH)

    @on_request(call_type=ApiType.FUNC)
    async def confess(self, message: str) -> str:
        return f"ζˆ‘ζ˜―εŽŸη₯žε“Ÿ! ζ”Άεˆ°ζ‚¨ηš„ζΆˆζ―{message}"

Cross-Plugin Access

FrameX allows a plugin to call APIs exposed by other plugins. This is useful for composition, reuse, and gradual migration across teams.

There are two steps to make a cross-plugin call:

  1. Declare dependencies in PluginMetadata.required_remote_apis.
  2. Invoke the remote API with self._call_remote_api(...) inside your plugin class.

1) Declare Required Remote APIs

Always declare what you will call in your plugin’s metadata. For example:

__plugin_meta__ = PluginMetadata(
    name="my-plugin",
    version="0.1.0",
    description="Demonstrates cross-plugin access",
    author="touale",
    url="http://example.local/FrameX",
    required_remote_apis=[
        "/api/v1/echo",  # HTTP (basic types)
        "/api/v1/echo_model",  # HTTP (BaseModel payload)
        "/api/v1/echo_stream",  # HTTP streaming
        "echo.EchoPlugin.confess",  # FUNC call
    ],
)

2) Call Remote APIs with self._call_remote_api(...)

Use self._call_remote_api(api_name, **kwargs) inside your plugin. The api_name is either:

  • The HTTP path (e.g., "/api/v1/echo"), or
  • The FUNC name (e.g., "echo.EchoPlugin.confess").

Arguments are passed as keyword arguments.

  • Primitive types (str/int/float/bool) β†’ pass directly.
  • Pydantic models β†’ pass a dict (e.g., model.model_dump() or a manually constructed dict).

2.1 HTTP (Basic Types)

Remote provider:

@on_request("/echo", methods=["GET"])
async def echo(self, message: str) -> str:
    return message

Caller usage:

result = await self._call_remote_api("/api/v1/echo", message=message)

2.2 HTTP (BaseModel Payload)

Remote provider:

class EchoModel(BaseModel):
    id: int
    name: str

@on_request("/echo_model", methods=["POST"])
async def echo_model(self, message: str, model: EchoModel) -> str:
    return f"{message},{model.model_dump()}"

Caller usage:

res = await self._call_remote_api(
    "/api/v1/echo_model",
    message=message,
    model={"id": 1, "name": "Genshin Impact"}  # BaseModel as dict
)

2.3 HTTP Streaming

Remote provider:

@on_request("/api/v1/echo_stream", methods=["GET"], stream=True)
async def echo_stream(self, message: str) -> AsyncGenerator[str, None]:
    for ch in f"Streaming fun, {message}":
        yield make_stream_event(StreamEnventType.MESSAGE_CHUNK, ch)
        await asyncio.sleep(0.1)
    yield make_stream_event(StreamEnventType.FINISH)

Caller usage:

stream = await self._call_remote_api("/api/v1/echo_stream", message=message)
stream_text = "".join([extract_content(evt) for evt in stream if "message_chunk" in evt])

2.4 FUNC Calls

@on_request(call_type=ApiType.FUNC)
async def confess(self, message: str) -> str:
    return f"Got your message: {message}"

Caller usage:

res = await self._call_remote_api("echo.EchoPlugin.confess", message=message)

3) How to Find api_name

An easy way to discover all available APIs is to check the startup logs, which list detected plugins and endpoints. Example:

09-04 15:10:54 [SUCCESS] framex.plugin.manage | Succeeded to load plugin "echo" from framex.plugins.echo
09-04 15:10:54 [INFO] framex | Start initializing all DeploymentHandle...
09-04 15:10:54 [SUCCESS] framex.plugin.manage | Found plugin HTTP API "/api/v1/echo" from plugin(hello_world)
09-04 15:10:54 [SUCCESS] framex.plugin.manage | Found plugin HTTP API "/api/v1/echo_stream" from plugin(hello_world)
09-04 15:10:54 [SUCCESS] framex.plugin.manage | Found plugin FUNC API "echo.EchoPlugin.confess" from plugin(echo)

4) End-to-End Example

import time
from typing import Any

from framex.consts import VERSION
from framex.plugin import BasePlugin, PluginMetadata, on_register, on_request, remote

__plugin_meta__ = PluginMetadata(
    name="invoker",
    version=VERSION,
    description="原η₯žδΌšθ°ƒη”¨θΏœη¨‹ζ–Ήζ³•ε“Ÿ",
    author="原η₯ž",
    url="https://github.com/touale/FrameX-kit",
    required_remote_apis=[
        "/api/v1/echo",
        "echo.EchoPlugin.confess",
        "/api/v1/echo_stream",
        "/api/v1/echo_model",
        
        
    ],
)


@on_register()
class InvokerPlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)


    @on_request("/evoke_echo", methods=["GET"])
    async def evoke(self, message: str) -> list[Any]:
        def extract_content(chunk: str) -> str:
            return chunk.split('"content": "')[-1].split('"')[0]

        echo = await self._call_remote_api("/api/v1/echo", message=message)
        stream = await self._call_remote_api("/api/v1/echo_stream", message=message)
        stream_text = "".join([extract_content(c) for c in stream if "message_chunk" in c])
        confess = await self._call_remote_api("echo.EchoPlugin.confess", message=message)
        echo_model = await self._call_remote_api(
            "/api/v1/echo_model", message=message, model={"id": 1, "name": "原η₯ž"}
        )
        return [echo, stream_text, confess, echo_model]

Plugin Configuration

FrameX supports TOML, and ENV (including .env) configuration formats and allows nested Pydantic models for strongly-typed settings.

We recommend TOML for multi-level configuration, as it is cleanly hierarchical and scales well when new plugin options are added.

1) System Config (Overview)

The runtime loads a top-level Settings model that typically includes:

  • server: ServerConfig β€” host/port, dashboard, use_ray, enable_proxy
  • log: LogConfig β€” log formatting and prefixes to ignore
  • sentry: SentryConfig β€” Sentry toggles, DSN, env (local|dev|prod), lifecycle
  • test: TestConfig β€” test switches
  • plugins: dict[str, Any] β€” plugin-specific raw config
  • load_builtin_plugins: list[str] β€” which built-in plugins to load (e.g. "proxy")
  • load_plugins: list[str] β€” which third-party plugins to load

The only configurations you need to focus on are server, load_plugins, load_builtin_plugins, and plugins. FrameX automatically manages everything else.

Example

By default, the runtime reads from config.toml at the project root. config.toml

# Load built-in and third-party plugins
load_builtin_plugins = ["proxy", "echo"]
load_plugins = ["your plugin"]

[server]
use_ray = false
enable_proxy = false
host = "127.0.0.1"
port = 8080

2) Plugin Config (typed)

Each plugin can define a dedicated typed config model and have it injected automatically at registration time.

1. Define a config model

class ProxyPluginConfig(BaseModel):
    proxy_urls: list[str] = []
    force_stream_apis: list[str] = []
    white_list: list[str] = []

2. Get it by get_plugin_config

from framex.plugin import get_plugin_config
settings = get_plugin_config({PLUGIN_NAME}, ProxyPluginConfig)

3) Where to Put Plugin Config

There are two supported locations. Choose one per plugin:

Add a sub-table under [plugins.<plugin_name>] in config.toml:

load_builtin_plugins = ["proxy"]

[server]
use_ray = false
enable_proxy = true

[plugins.proxy]
proxy_urls = ["http://127.0.0.1:8080"]
force_stream_apis = ["/api/v1/chat"]

Plan B) Make it in .env or env environment

For example in .env:

server__use_ray=false
server__enable_proxy=true
plugins__proxy__proxy_urls=["http://127.0.0.1:8080"]
plugins__proxy__force_stream_apis=["/api/v1/chat"]

Note:

  • Nested keys are flattened using double underscores (__).
  • Configuration keys should be written in lowercase. Uppercase keys (e.g. server__USE_RAY) will not be recognized by Pydantic in this setup.

4) Supported Formats & Loading Order

Supported sources (from highest to lowest precedence):

  1. ENV settings (process environment variables)
  2. dotenv file (e.g., .env)
  3. Project root config.toml (global TOML)
  4. pyproject.toml (project-level fallback)

Recommendation: Prefer TOML for hierarchical configuration. It is expressive, diff-friendly, and scales well as plugins evolve.

Plugin Loading & Startup

FrameX supports two ways to load plugins:

  1. Programmatically (in code)
  2. Declaratively via configuration (config.toml or other supported formats)

Both approaches end with starting the runtime using framex.run().

1) Programmatic Loading

Import framex in your application entrypoint and load plugins explicitly.

def main() -> None:
    import framex

    # 1) Load built-in plugins (shipped with FrameX)
    framex.load_builtin_plugins("echo")  # system/built-in plugin

    # 2) Load a single plugin (module or package)
    # framex.load_plugins("your_project.plugins.plugin_name")
    # βœ… Example:
    # framex.load_plugins("demo_project.plugins.hello_world")

    # 3) Load multiple plugins at once
    # framex.load_plugins("xxx.plugin_a", "xxx.plugin_b")
    # βœ… Example:
    # framex.load_plugins(
    #     "demo_project.plugins.hello_world",
    #     "demo_project.plugins.play_games",
    # )

    # 4) Load an entire plugins package (recommended)
    #    This recursively discovers all valid plugins under the `plugins/` directory.
    #    Every discovered module/package must define `__plugin_meta__`.
    # framex.load_plugins("your_project.plugins")
    # framex.load_plugins("your_project")

    # Finally, start the runtime
    framex.run()

Notes:

  • Each discovered module/package must declare plugin_meta = PluginMetadata(...).

2) Configuration-Based Loading

You can specify which plugins to load from your config (e.g., config.toml). This is the simplest way to manage environments and deployments.

# Built-in plugins (shipped with FrameX)
load_builtin_plugins = ["proxy"]

# Third-party or app plugins (python import paths)
load_plugins = ["someone.plugins"]

At startup, FrameX reads the config (see Plugin Configuration section for formats and precedence), automatically loads the declared plugins, and then starts the service with framex.run().

What these fields mean

  • load_builtin_plugins β€” a list of built-in (platform) plugins to enable (e.g., "proxy").
  • load_plugins β€” a list of user/application plugin import paths:
  • Single module plugin: "demo_project.plugins.hello_world"
  • Package plugin (folder): "demo_project.plugins"
  • Multiple entries allowed.

3) Discovery Rules & Requirements

  • Metadata: Every plugin module/package must define plugin_meta = PluginMetadata(...).
  • Layout: Both single-file (plugins/foo.py) and folder-based plugins (plugins/bar/) are supported.
  • Errors: If a plugin fails validation, FrameX will log a clear error and skip it.

Plugin Debugging & Testing

Debugging FrameX plugins is straightforward. When Ray is disabled ([server].use_ray = false), your plugin code runs in a standard FastAPI app, so you can set breakpoints and debug just like any regular Python service.

1) Debugging

  1. Disable Ray in your config.toml:
[server]
use_ray = false
  1. Start the app normally (e.g., framex.run()).
  2. Use your IDE’s debugger to set breakpoints anywhere in your plugin handlers (@on_request) or lifecycle hooks (init, on_start).

With Ray disabled, there’s no difference from debugging a standard FastAPI application.

2) Testing

FrameX integrates naturally with pytest and fastapi.testclient. You can run the app in test mode and exercise your plugin’s HTTP endpoints and streaming behavior.

Pytest Fixtures(consts.py)

import pytest
from typing import Generator
from fastapi import FastAPI
from fastapi.testclient import TestClient
import framex

@pytest.fixture(scope="session", autouse=True)
def test_app() -> FastAPI:
    # Boot the FrameX app in test mode (no Ray, in-memory app)
    return framex.run(test_mode=True)  # type: ignore[return-value]

@pytest.fixture(scope="session")
def client(test_app: FastAPI) -> Generator[TestClient, None, None]:
    with TestClient(test_app) as c:
        yield c

Example Tests

import json
from fastapi.testclient import TestClient
from framex.consts import API_STR

def test_echo(client: TestClient):
    params = {"message": "hello world"}
    res = client.get(f"{API_STR}/echo", params=params).json()
    assert res["status"] == 200
    assert res["data"] == params["message"]

def test_echo_model(client: TestClient):
    params = {"message": "hello world"}
    data = {"id": 1, "name": "原η₯ž"}
    res = client.post(f"{API_STR}/echo_model", params=params, json=data).json()
    assert res["status"] == 200
    assert res["data"] == "hello world,{'id': 1, 'name': '原η₯ž'}"

def test_echo_stream(client: TestClient):
    params = {"message": "hello world"}
    # Server-Sent Events (SSE) style stream
    with client.stream("GET", f"{API_STR}/echo_stream", params=params) as res:
        assert res.status_code == 200
        chunks = []
        events = set()

        for line in res.iter_lines():
            if not line:
                continue
            if line.startswith("event: "):
                events.add(line.removeprefix("event: "))
            elif line.startswith("data: "):
                js = json.loads(line.removeprefix("data: "))
                content = js.get("content")
                if content:
                    chunks.append(content)

        assert events == {"finish", "message_chunk"}
        assert "".join(chunks) == f"原η₯žηœŸε₯½ηŽ©ε‘€, {params['message']}"

Advanced Usage

This section covers advanced features of FrameX that go beyond the basic plugin lifecycle.
While the basic usage allows you to register, configure, and run plugins easily, advanced usage focuses on scalability, compatibility, and resilience in production systems.

Topics

  • System Proxy Plugin (v2 API Compatibility)
    Learn how to use the built-in proxy plugin to forward requests to legacy v2 APIs.
    This ensures smooth migration from v2 to v3 without breaking compatibility, and provides a zero-code-change transition once an algorithm is upgraded.

  • Integrating Ray Engine
    Switch the execution backend to Ray with a single configuration change.
    Gain distributed execution, high concurrency, and fault isolation, ideal for production workloads.

  • Advanced Remote Calls & Non-Blocking Execution
    Mark blocking functions with @remote to offload them into distributed, non-blocking execution.
    Prevent plugin-level blocking while keeping your code simple and async-friendly.

  • Monitoring & Tracing
    Integrate with Sentry to capture errors, monitor performance, and trace requests across plugins.
    Provides full observability for both development and production environments.

Why Advanced Usage Matters?

  • πŸ”„ Compatibility: Seamlessly migrate from v2 APIs.
  • ⚑ Performance: Handle large-scale workloads with Ray.
  • πŸ›‘οΈ Resilience: Avoid blocking with distributed remote calls.
  • πŸ“Š Observability: Gain insights into runtime behavior with tracing and monitoring.

These features are optional for development but essential for production environments where scalability and stability are critical.

System Proxy Plugin: API Compatibility with Regular FastAPI Projects and Other FrameX Instances

Its goal is to support progressive adoption of FrameX β€” allowing any unmigrated or legacy algorithms to be automatically forwarded to external services or other FrameX instances during the transition, ensuring compatibility and service stability.

In addition, the proxy can forward cross-plugin requests across different FrameX instances, including remote instances, enabling distributed execution and transparent inter-instance communication within the same unified API interface.

Why use the Proxy?

  • Gradual migration β€” Maintain legacy services while progressively adopting FrameX.
  • API compatibility β€” Expose existing endpoints through FrameX with minimal integration effort.
  • Zero code intrusion β€” Invoke remote or legacy APIs using the same _call_remote_api(...) interface.
  • Streaming support β€” Enable selective streaming behavior via force_stream_apis.

config.toml

load_builtin_plugins = ["proxy"]

[server]
enable_proxy = true

[plugins.proxy]
proxy_urls = ["http://127.0.0.1:80"]
force_stream_apis = ["/api/v1/chat"]

The proxy automatically discovers available APIs (for example, through OpenAPI introspection) and dynamically maps them to corresponding call signatures within FrameX.

For any streaming endpoints defined in force_stream_apis, the proxy automatically handles responses as Server-Sent Events (SSE) or chunked data streams, depending on the endpoint’s behavior.

Calling Legacy APIs from FrameX

You can easily invoke legacy APIs (e.g., from existing FastAPI services or other FrameX instances) through the System Proxy Plugin. Simply declare the remote APIs in your plugin metadata, and call them using the _call_remote_api(...) helper. Basic types can be passed directly, while Pydantic models should be converted to dict form.

__plugin_meta__ = PluginMetadata(
    name="invoker",
    version=VERSION,
    description="Invoke external APIs via the system proxy",
    author="touale",
    url="https://github.com/touale/FrameX-kit",
    required_remote_apis=[
        "/api/v1/base/match", # Define your dependent APIs here
    ],
)

# Simple call (non-streaming)
remote_version = await self._call_remote_api("/api/v1/base/version")

# Call with body payload (Pydantic -> dict)
match_result = await self._call_remote_api(
    "/api/v1/base/match",
    model={
        "name": "test",
    },
)

That’s it β€” the proxy automatically forwards your call to the corresponding service, adapts the request/response formats when necessary, and returns the result to your plugin seamlessly.

Streaming Endpoints

Add any streaming paths to force_stream_apis so the proxy treats them as streaming:

[plugins.proxy]
proxy_urls = ["http://127.0.0.1:80"]
force_stream_apis = ["/api/v1/chat", "/api/v1/echo_stream"]

Then, when you call those endpoints via _call_remote_api(...), you’ll receive an async iterable (or SSE lines) that you can consume incrementally in v3.

Configuration Reference

[server]
enable_proxy = true      # Enable the proxy bridge

[plugins.proxy]
proxy_urls = ["http://<host>:<port>", "..."]   # One or more upstream API endpoints (supports load balancing)
force_stream_apis = ["/api/v1/chat"]           # Endpoints treated as streaming

# Optional filters:
# white_list = ["/api/v1/*"]                   # Whitelisted API paths (restricts to these only)

Transparent Migration (Zero-Code Changes)

One of the key advantages of the Proxy Plugin design is its support for transparent migration:

  • Once an API is declared in required_remote_apis and invoked through _call_remote_api(...), it does not matter where the actual implementation resides β€” whether in a legacy service or another FrameX instance.
  • When that implementation is later migrated into the current FrameX environment as a native plugin, the framework will automatically route calls to the new local version.
  • Your plugin code remains completely unchanged β€” no modification to call sites or configuration is required.

This ensures a smooth transition path where business logic and integrations stay stable while the backend evolves naturally.

Integrating Ray Engine

FrameX supports multiple execution backends. To switch to Ray for distributed execution, you don’t need to change any code β€” simply turn it on in the configuration.

Recommendation:

  • Keep use_ray = false during day-to-day development and local debugging.
  • Enable Ray only in production (or staging) where distributed execution improves throughput and tail latency.

Quick Start

Enable Ray in your config.toml:

[server]
use_ray = true             # ← toggle on Ray

No code changes are required. Your existing plugins, @on_request(...) handlers, and cross-plugin calls will continue to work. FrameX will place plugin deployments on Ray actors and route requests accordingly.

What’s Ray?

Ray simplifies distributed computing by providing:

  • Scalable compute primitives: Tasks and actors for painless parallel programming
  • Specialized AI libraries: Tools for common ML workloads like data processing, model training, hyperparameter tuning, and model serving
  • Unified resource management: Seamless scaling from laptop to cloud with automatic resource handling

When to Use Ray

  • High concurrency / throughput workloads
  • CPU/GPU intensive algorithms
  • Horizontal scaling across multiple nodes
  • Background or non-blocking long-running tasks (see Distributed Remote Calls)

Keep it disabled for:

  • Rapid iteration, local debugging, or stepping through code (simpler without Ray)
  • Unit testing (use framex.run(test_mode=true))

Why Ray?

Enabling Ray gives FrameX the ability to:

  • πŸš€ Boost performance with high concurrency and high throughput distributed computation.
  • 🧩 Isolate blocking plugins so that if one plugin experiences latency or heavy computation, it won’t block other plugins from responding.
  • πŸ“¦ Distribute heavy workloads (e.g., model inference, batch computation) across multiple nodes, avoiding single-node bottlenecks.
  • πŸ”„ Offload blocking tasks to Ray’s distributed task scheduler, preventing API endpoints from hanging and improving responsiveness.
  • βš–οΈ Scale elastically β€” add more Ray workers to handle increasing workloads without code changes.

This makes Ray especially suitable for production environments running large-scale algorithmic systems.

Advanced Remote Calls & Non-Blocking Execution

Even though enabling Ray ensures that one plugin’s heavy workload will not block other plugins, it does not prevent blocking inside a single plugin itself.

If some part of your plugin code performs a blocking operation (e.g., time.sleep, long-running computation, or a non-async library call), the entire plugin instance may become unresponsive.

FrameX provides the @remote decorator to solve this issue:

  • it offloads the decorated function to distributed execution (via Ray when enabled, or a fallback mechanism otherwise), making it non-blocking by design.

1) How It Works

  • Add @remote() to a method (sync or async).
  • Call it with .remote(...) instead of normal invocation.
  • FrameX automatically executes the method in a separate worker (via Ray when available).
  • Your plugin remains responsive, even if the method blocks.

2) Supported Function Types

The @remote decorator is fully compatible with:

  • Regular (synchronous) functions
  • Asynchronous (async) functions
  • Instance methods (with self)
  • Static methods and class methods

3) Example Highlights

1. Add @remote() to a method.

(a). Blocking synchronous function

@remote()
def remote_sleep():
    time.sleep(0.1)
    return "remote_sleep"

(b). Asynchronous function

@remote()
async def remote_func_async_with_params(a: int, b: str):
    return f"{a}, {b}"

(c). Class instance method

class Worker:
    @remote()
    def heavy_compute(self, n: int):
        return sum(i * i for i in range(n))

(d). Static method

class Utils:
    @remote()
    @staticmethod
    def convert(data: str) -> str:
        time.sleep(1)
        return data.upper()

2. Call it with .remote(...) instead of normal invocation.

results = [
    await remote_sleep.remote(),
    await remote_func_async_with_params.remote(123, "abc"),
    await Worker().heavy_compute.remote(10000),
    await Utils.convert.remote("framex"),
]

3) Key Benefits

  • βœ… Prevents plugin self-blocking.
  • βœ… Works with both sync and async methods.
  • βœ… Transparent: no code changes needed when Ray is disabled/enabled.
  • βœ… Simple syntax: .remote(...) for non-blocking execution.

With @remote, you can safely use blocking libraries, legacy synchronous code, or heavy computations inside FrameX plugins, while still keeping your APIs responsive and scalable.

Authentication & Authorization

This section describes how to configure documentation authentication and API authentication in the system, including proxy-related authorization for remote services.

1) Documentation Authentication

The system supports HTTP Basic Authentication for built-in API documentation pages.

Protected Endpoints

The following endpoints are protected when documentation authentication is enabled:

  • /docs
  • /redoc
  • /api/v1/openapi.json

Configuration

Documentation authentication is configured in config.toml:

[server]
docs_user = "admin"
docs_password = "admin"

Default Behavior

  • docs_user

    • Default value: admin

  • docs_password

    • If not set or left empty, the system uses the default password: admin

Authentication Failure

  • HTTP Status Code: 401 Unauthorized

2) API Authentication (Rules-Based)

The system supports API-level authentication using access keys, configured through a rules-based authorization model.

Overview

  • Authentication is defined by a single rules mapping
  • Each rule maps an API path to a list of allowed access keys
  • Only URLs explicitly defined in rules are protected
  • URL matching supports:
    • Exact match
    • Prefix wildcard match using /*
  • For wildcard rules, the longest matching prefix wins

Configuration

[auth]
rules = {"/api/v1/echo_model" = ["key-1", "key-2"],"/api/v2/*" = ["key-3"]}

Runtime Behavior

  • If a request URL does not match any rule, authentication is not required
  • If a request URL matches a rule, a valid access key must be provided
  • Missing or invalid keys result in:
    • HTTP Status Code: 401 Unauthorized

3) Proxy Plugin Authentication

The proxy plugin uses the same rules-based authentication mechanism as standard API authentication.

Configuration

[plugins.proxy.auth]
rules = {"/api/v1/proxy/remote" = ["proxy-key"],"/api/v1/echo_model" = ["echo-key"]}

Concurrency & Ingress Configuration

This chapter introduces how to configure concurrency, scaling, and ingress behavior in FrameX using Ray.
By tuning these parameters, you can control instance count, request concurrency, and overall throughput.

1) Overview

FrameX supports additional Ray Serve ingress configurations to improve:

  • Maximum concurrent requests
  • Request queueing behavior
  • Instance-level load control

These configurations are applied through ingress_config and can be defined at different levels with clear inheritance rules.

This chapter uses max_ongoing_requests as an example.

2) Base Ingress Configuration

FrameX provides a global base configuration:

base_ingress_config = {"max_ongoing_requests" = 10}

Behavior

  • Acts as the default ingress configuration
  • Automatically inherited by:
    • Server
    • All plugins
  • Can be overridden at lower levels

3) Server-Level Ingress Configuration

You can override the base configuration at the server level:

[server]
ingress_config = {"max_ongoing_requests" = 60}

Behavior

  • Applies to server-level ingress
  • Overrides base_ingress_config

4) Plugin-Level Ingress Configuration

Plugins can define their own ingress configuration.

Example for the proxy plugin:

[plugins.proxy]
ingress_config = {"max_ongoing_requests" = 60}

Behavior

  • Takes precedence over both:
    • server.ingress_config
    • base_ingress_config
  • Only applies to the specific plugin

5) Plugin Development: Custom Ingress Configuration

If a plugin requires a custom max_ongoing_requests or other Ray Serve parameters, follow these steps.

1. Add ingress_config to Plugin Config

from pydantic import BaseModel
from typing import Any


class ExamplePluginConfig(BaseModel):
    ingress_config: dict[str, Any] = {"max_ongoing_requests": 60}

2. Apply ingress_config During Registration

Use the on_register decorator to inject ingress parameters:

@on_register(**settings.ingress_config)
class ExamplePlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None: ...

Behavior

  • settings.ingress_config is passed directly to Ray Serve
  • Allows fine-grained control per plugin
  • Fully compatible with Ray Serve autoscaling and concurrency features

6) Configuration Inheritance Rules

Ingress configuration follows a top-down inheritance model:

  1. base_ingress_config
  2. plugins.<plugin_name>.ingress_config

If a plugin does not define ingress_config, it automatically inherits from the nearest parent level.

Complete Example: config.toml

base_ingress_config = {"max_ongoing_requests" = 10}

[server]
ingress_config = {"max_ongoing_requests" = 60}

[plugins.proxy]
ingress_config = {"max_ongoing_requests" = 60}

7) Supported Ray Serve Parameters

In addition to max_ongoing_requests, Ray Serve supports many advanced parameters such as:

  • Autoscaling behavior
  • Replica scaling limits
  • Request queue management

For the complete and up-to-date list, refer to the official Ray Serve documentation:

https://docs.rayai.org.cn/en/latest/serve/advanced-guides/advanced-autoscaling.html

Proxy Function & Remote Invocation

This chapter introduces the proxy function mechanism in FrameX, including a new decorator, runtime support, and an HTTP endpoint for remote proxy invocation.

This feature enables cross-FrameX-instance function execution, allowing plugins to transparently call functions hosted on remote FrameX instances.

1) Background & Motivation

In multi-FrameX deployments, different instances may run under different security or infrastructure constraints.

Typical scenarios include:

  • Instance A does not have access to MySQL
  • Instance A is restricted to HTTP-only outbound access
  • Instance B has database access or privileged network permissions

To fully implement plugin functionality without violating security constraints, FrameX introduces the on_proxy mechanism, enabling remote function proxying across FrameX instances.

2) Design Overview

  • Functions can be marked as proxy-enabled
  • FrameX automatically decides whether to execute locally or remotely
  • No explicit client/server role configuration is required
  • The same code runs on both sides

3) Defining a Proxy Function

Step 1: Register Proxy Function in Plugin

Proxy functions must be registered during plugin startup.

Important: Lazy import is required.

@on_register()
class ExamplePlugin(BasePlugin):
    def __init__(self, **kwargs: Any) -> None:
        super().__init__(**kwargs)

    async def on_start(self) -> None:
        from demo.some import example_func  # Important: Lazy import

        await register_proxy_func(example_func)

Step 2: Mark Function with on_proxy

@on_proxy()
@other_func
async def example_func(id, user):
    data = "..."
    return data

4) Enabling Remote Proxy Invocation

To enable remote execution, add the following configuration to config.toml.

Proxy Configuration

[plugins.proxy]
proxy_urls = ["http://remotehost:8080"]
white_list = ["/api/v1/proxy/remote"]
proxy_functions = {"http://remotehost:8080" = ["demo.some.example_func"]}

Authentication Configuration

[plugins.proxy.auth]
rules = {"/api/v1/proxy/remote" = ["proxy-key"],"/api/v1/openapi.json" = ["openapi-key"]}

Once configured, FrameX automatically routes function calls to remote instances when required.

5) Security, Performance & Compatibility

Security

  • Automatic serialization and deserialization
  • Data compression and encryption
  • Safe fallback when local decorators fail

Compatibility

Supports:

  • Primitive types
  • BaseModel (Pydantic)
  • Most stateless class objects

Simulating Network Communication (For Tests)

This feature is designed exclusively for tests.
When writing unit tests or integration tests that involve HTTP calls, you should avoid calling real external services.
Instead, use pytest + VCR to record the first HTTP interaction and replay it in subsequent test runs.

1) Why?

  • Makes tests deterministic (no dependency on unstable external APIs).
  • Makes tests faster by avoiding repeated network requests.
  • Prevents API quota exhaustion or accidental use of sensitive credentials.
  • Ensures CI pipelines run consistently with no external dependencies.

2) VCR Configuration (for tests)

In your tests/conftest.py, define fixtures for pytest:

@pytest.fixture(scope="module")
def vcr_config():
    return {
        "record_mode": "new_episodes",  # record new calls only when missing
        "filter_headers": ["authorization", "api-key"],  # hide secrets
        "ignore_hosts": ["testserver"],  # don't record FastAPI TestClient
        "match_on": ["uri", "method", "body", "path", "query"],
        "allow_playback_repeats": True,
    }

@pytest.fixture(scope="module")
def disable_recording(request):  # noqa
    return settings.test.disable_record_request

def before_record_request(request):
    # Skip recording noisy or irrelevant endpoints
    if all(ch not in request.path for ch in ["rerank", "minio"]):
        return request
    return None

def before_record_response(response):
    if response["status"]["code"] != 200:
        return None
    return response

This ensures only test-related network traffic is recorded and sensitive headers are stripped out.

3) Writing Tests with VCR

Annotate your test with @pytest.mark.vcr(...) in need:

import pytest
from fastapi.testclient import TestClient

from framex.consts import API_STR
from tests.conftest import before_record_request, before_record_response

@pytest.mark.vcr(before_record_request=before_record_request, before_record_response=before_record_response)
def test_get_proxy_version(client: TestClient):
    res = client.get(f"{API_STR}/base/version").json()
    assert res["status"] == 200
    assert res["data"]

The first run will auto record the request and save it into a cassette file under tests/cassettes/. Later runs will replay the response from the cassette, with no real network call.

Monitoring & Tracing

FrameX integrates both Sentry and Ray Dashboard for comprehensive monitoring and tracing. This enables you to capture errors, exceptions, performance metrics, task status, and distributed resource usage across the entire system with minimal setup.

Benefits

  • βœ… Automatic error capturing (no manual try/except needed).
  • βœ… Performance tracing (slow APIs, blocked tasks, async bottlenecks).
  • βœ… Centralized monitoring across multiple plugins.
  • βœ… Distributed resource and task monitoring via Ray Dashboard.
  • βœ… Flexible configuration per environment.

1) Quick Setup

Sentry Configuration

Simply add a [sentry] section to your configuration file (config.toml):

[sentry]
enable = true
dsn = "<your-sentry-dsn>"
env = "local"        # e.g., local, dev, prod
debug = true
ignore_errors = []
lifecycle = "trace"
integration = []
enable_logs = true
  • enable: Toggles Sentry integration on/off.
  • dsn: Your Sentry DSN (Data Source Name).
  • env: Environment tag for grouping events (e.g. local, dev, prod).
  • debug: Prints debug logs for Sentry itself.
  • ignore_errors: A list of errors to ignore.
  • lifecycle: Mode of event tracking (manual or trace).
  • enable_logs: Captures logs in addition to errors.

Ray Dashboard Configuration

To enable distributed monitoring, configure the [server] section:

[server]
dashboard_host = "127.0.0.1"
dashboard_port = 8260
use_ray=true

Once Ray is running, the dashboard is available at:

http://127.0.0.1:8260

Note that you need to switch the engine to Ray.

Development Guide

This document outlines the branching strategy and commit message guidelines for the project.
Please follow these rules to ensure smooth collaboration and maintainability.

1) Develop Project

  1. Setup Your Environment: Before you start making changes, set up your local development environment. Run:
pip install uv && uv sync --dev
  1. Test the Server: Ensure the server runs correctly on your local machine:
framex run --load-builtin-plugins echo

3、Making Changes：

  • Start Coding: Once you’ve confirmed everything is set up correctly, you can start coding. Make sure to work on a new branch created from the latest main or dev branch.
  • Validate Your Changes: After making your changes, ensure all tests pass:
poe test-all

2) How to Manage a Package with uv

To add a new package to the project, use the uv add command. This will update the pyproject.toml file and install the new dependency.

Adding a Regular Dependency

To add a package as a regular dependency, run uv add with the package name:

uv add requests

Adding a Development Dependency

To add a package to a specific group like dev, use the --group flag:

uv add pytest –-group dev

Remove a Dependency

To remove a package, use the uv remove command:

uv remove requests

Update a specific dependency package

Use the uv lock --upgrade-package <package> command. For example, to upgrade sqlmodel:

uv lock --upgrade-package sqlmodel
uv sync # or `uv sync --dev`

3) Poe Help

Use poe help to get more help.

USAGE
  poe [-h] [-v | -q] [-C PATH] [--ansi | --no-ansi] task [task arguments]

CONFIGURED TASKS

  style                 Auto-fix and format the code using ruff.
  lint                  Check for style and type violations using ruff and mypy.
  clean                 Remove all files generated by builds and tests.
  test                  Run tests quickly using the default Python environment.
  test-all              Run all tests.
  test-ci               Run ci tests.
  build                 Build source and wheel package
  tag                   Create a new version tag
  publish               Publish the package to PyPI
  release               Build source and wheel package

4) About Branches

Master (master | main)

  • The master branch is the stable version of the project.
  • It contains the official releases ready for production.
  • Only administrators are allowed to merge into master.

Development (dev)

  • The dev branch is used for day-to-day development.
  • New features and bug fixes should be merged into dev.
  • Beta versions are managed and released from the dev branch for testing before promotion to master.

Personal / Feature Branches

  • Developers should create a personal branch or feature branch:
    • Personal branch: named after the developer, e.g., username.
    • Feature branch: named with a clear prefix, e.g., feat-login, fix-bug-xxx.
  • When development is complete, submit a Pull Request (PR) targeting the dev branch for review.

5) Commit Message Guidelines

All commits must follow the format:

<tag>: <Message>

Allowed Tags

  • feat: New features
  • fix: Bug fixes
  • build: Build process or dependency changes
  • chore: Routine tasks with no code changes (e.g., configs)
  • ci: Continuous Integration / pipeline changes
  • docs: Documentation updates
  • perf: Performance improvements
  • style: Code style changes (formatting, whitespace, etc.)
  • refactor: Code refactoring without functional changes
  • test: Adding or updating tests
  • update: Minor updates or improvements

Example

feat: add support for plugin configuration loading
fix: resolve bug in cross-plugin API call
docs: update README with setup instructions

6) Versioning Rules

  • Minor version bumps

    • Triggered by commits tagged with: update.
    • Represents major feature additions, significant enhancements, or a large batch of changes.
    • Example: if the current version is 1.1.0, the next release will be 1.2.0.

  • Patch version bumps

    • Triggered by commits tagged with: feat, fix, perf, refactor, build.
    • feat: small/new feature additions.
    • fix: bug fixes.
    • perf: performance improvements.
    • refactor: internal code refactoring.
    • build: dependency/build-related adjustments.
    • Example: if the current version is 1.1.0, the next release will be 1.1.1.

Examples

  • Current version: 1.1.0
    • A commit with update: improve plugin loading speed β†’ new version 1.2.0
    • A commit with fix: resolve proxy plugin crash β†’ new version 1.1.1

By following these rules, the team ensures consistency, easier reviews, and automated version management.

FAQ & Troubleshooting

1. Virtual environment not auto-activated after uv sync --dev

Issue:
After running uv sync --dev, the virtual environment (.venv) is created but not automatically activated.

Solution:
Manually activate the environment:

source .venv/bin/activate

2. mypy reports missing type stubs for third-party libraries

$ poe lint
Poe => ruff check . --fix
All checks passed!
Poe => mypy .
src/find_policy/__init__.py:7: error: Skipping analyzing "xxxxxx": module is installed, but missing library stubs or py.typed marker  [import-untyped]
src/find_policy/__init__.py:7: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 12 source files)
Error: Sequence aborted after failed subtask '_lint'

Solutions:

(1). (Not recommended) Tell mypy to ignore missing stubs by adding to mypy.ini or pyproject.toml:

For example mindforge:

[mypy-mindforge.*]
ignore_missing_imports = True

(2). (Recommended) Install the corresponding type stub package if available.

For example pytz pyyaml:

uv add types-pytz --dev     # for pytz
uv add types-pyyaml --dev   # for pyyaml