Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 114 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,18 +37,24 @@

### For Data Scientists
- **Project Management** -- Organize experiments with stage-based workflow (Ideation, Development, Production)
- **Project-Scoped Filtering** -- Global project selector in the topbar scopes every page (models, datasets, experiments, jobs, workspaces, features, visualizations) to a single project
- **Model Editor** -- Write and edit models directly in the browser with Monaco (Python + Rust)
- **Real-Time Training** -- Watch loss curves update live via SSE during training
- **Model Registry & CLI** -- Search, install, and manage models from the [Open Model Registry](https://github.com/GACWR/open-model-registry) via CLI (`openmodelstudio install iris-svm`) or the in-app registry browser. Install status syncs bidirectionally between CLI and UI
- **Real-Time Training** -- Watch loss curves, accuracy, and all metrics auto-update live during training with second-level duration accuracy
- **Generative Output Viewer** -- See video/image/audio outputs as models train
- **Experiment Tracking** -- Compare runs with parallel coordinates and sortable tables
- **JupyterLab Workspaces** -- Launch cloud-native notebooks with one click
- **Visualizations & Dashboards** -- 9 visualization backends (matplotlib, seaborn, plotly, bokeh, altair, plotnine, datashader, networkx, geopandas) with a unified `render()` abstraction. Combine visualizations into drag-and-drop dashboards with persistent layout
- **Global Search** -- Cmd+K command palette searches across models, datasets, experiments, training jobs, projects, and visualizations with instant navigation
- **Notifications** -- Real-time notification bell with unread count, grouped timeline (Today / This Week / Earlier), mark-all-read, and context-aware icons
- **JupyterLab Workspaces** -- Launch cloud-native notebooks pre-loaded with tutorial notebooks (Welcome, Visualizations, Registry)
- **LLM Assistant** -- Natural language control of the entire platform
- **AutoML** -- Automated hyperparameter search
- **Feature Store** -- Reusable features across projects

### For ML Engineers
- **Kubernetes-Native** -- Every model trains in its own ephemeral pod
- **Rust API** -- High-performance backend built with Axum + SQLx
- **Python SDK & CLI** -- `pip install openmodelstudio` gives you both a Python SDK (`import openmodelstudio as oms`) and a CLI for registry management, model install/uninstall, and configuration
- **GraphQL** -- Auto-generated from PostgreSQL via PostGraphile
- **Streaming Data** -- Never load full datasets to disk
- **One-Command Deploy** -- `make k8s-deploy` sets up everything
Expand All @@ -68,6 +74,60 @@
<img src="docs/screenshots/oms-screenshot2.png" alt="OpenModelStudio Workspaces and Model Metrics" width="100%" />
</p>

### Visualizations & Dashboards

Create, render, and publish data visualizations from notebooks or the in-browser editor. OpenModelStudio supports **9 visualization backends** with a unified `render()` function that auto-detects the library:

| Backend | Output | Use Case |
|---------|--------|----------|
| matplotlib | SVG | Standard plots, publication-quality figures |
| seaborn | SVG | Statistical visualization, heatmaps |
| plotly | JSON | Interactive charts with zoom, pan, hover |
| bokeh | JSON | Interactive streaming charts |
| altair | JSON | Declarative Vega-Lite specifications |
| plotnine | SVG | ggplot2-style grammar of graphics |
| datashader | PNG | Server-side rendering for millions of points |
| networkx | SVG | Network/graph visualizations |
| geopandas | SVG | Geospatial maps |

```python
import openmodelstudio as oms

viz = oms.create_visualization("loss-curve", backend="plotly")
output = oms.render(fig, viz_id=viz["id"]) # auto-detects backend
oms.publish_visualization(viz["id"]) # available for dashboards
```

Combine visualizations into **drag-and-drop dashboards** with resizable panels, lock/unlock layout, and persistent configuration. Each visualization also has a full **in-browser editor** (`/visualizations/{id}`) with Monaco, live preview for JSON backends, template insertion, and data/config tabs.

<p align="center">
<img src="docs/screenshots/oms-screenshot3.png" alt="OpenModelStudio Visualization Framework" width="100%" />
</p>

### Model Registry

Browse, install, and manage models from the [Open Model Registry](https://github.com/GACWR/open-model-registry) -- a public GitHub repo that acts as a decentralized model package manager.

**From the CLI:**
```bash
openmodelstudio search classification # Search by keyword
openmodelstudio install iris-svm # Install a model
openmodelstudio list # List installed models
```

**From a notebook or script:**
```python
import openmodelstudio as oms

iris = oms.use_model("iris-svm") # Load from registry
handle = oms.register_model("my-iris", model=iris) # Register in project
job = oms.start_training(handle.model_id, wait=True) # Train it
```

`use_model()` resolves via the platform API, so it works inside workspace containers (K8s pods) without filesystem access. If the model isn't installed yet, it auto-installs from the registry. The web UI registry page shows **Installed** / **Not Installed** badges that stay in sync with CLI operations.

---

## Quick Start

### Prerequisites
Expand Down Expand Up @@ -142,9 +202,9 @@ This will:
| **Frontend** | Next.js 16, shadcn/ui, Tailwind, Recharts | App Router, Monaco editor, SSE streaming, Cmd+K search |
| **API** | Rust, Axum, SQLx | JWT auth, RBAC, K8s client, SSE metrics, LLM integration |
| **PostGraphile** | Node.js | Auto-generated GraphQL from PostgreSQL schema |
| **PostgreSQL 16** | SQL | Primary data store: users, projects, models, jobs, datasets, experiments |
| **PostgreSQL 16** | SQL | Primary data store: users, projects, models, jobs, datasets, experiments, visualizations, dashboards, notifications |
| **Model Runner** | Python/Rust | Ephemeral K8s pods per training job, streaming metrics |
| **JupyterHub** | Python | Per-user JupyterLab with pre-configured SDK and datasets |
| **JupyterHub** | Python | Per-user JupyterLab with pre-configured SDK, tutorial notebooks, and datasets |

### Training Job Lifecycle

Expand All @@ -161,15 +221,18 @@ User clicks "Train" --> API creates training_job record
### Database Schema (Key Tables)

```sql
users (id, email, name, password_hash, role, created_at)
projects (id, name, description, stage, owner_id, created_at)
models (id, project_id, name, framework, created_at)
model_versions (id, model_id, version, code, created_at)
jobs (id, project_id, model_id, job_type, status, config, metrics, started_at, completed_at)
datasets (id, project_id, name, path, format, size_bytes, created_at)
experiments (id, project_id, name, description, created_at)
experiment_runs (id, experiment_id, parameters, metrics, created_at)
workspaces (id, user_id, status, jupyter_url, created_at)
users (id, email, name, password_hash, role, created_at)
projects (id, name, description, stage, owner_id, created_at)
models (id, project_id, name, framework, registry_name, created_at)
model_versions (id, model_id, version, code, created_at)
jobs (id, project_id, model_id, job_type, status, config, metrics, started_at, completed_at)
datasets (id, project_id, name, path, format, size_bytes, created_at)
experiments (id, project_id, name, description, created_at)
experiment_runs (id, experiment_id, parameters, metrics, created_at)
workspaces (id, user_id, status, jupyter_url, created_at)
visualizations (id, project_id, name, backend, code, output_type, output_data, published, created_at)
dashboards (id, project_id, name, description, layout, created_at)
notifications (id, user_id, title, message, notification_type, read, link, created_at)
```

> See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full architecture documentation.
Expand All @@ -181,7 +244,9 @@ workspaces (id, user_id, status, jupyter_url, created_at)
Follow these guides to go from zero to a fully tracked ML experiment:

1. **[Usage Guide](docs/USAGE.md)** -- Log in, create a project, upload a dataset, launch a workspace
2. **[Modeling Guide](docs/MODELING.md)** -- Train, evaluate, and track models using the SDK (13-cell notebook walkthrough)
2. **[Modeling Guide](docs/MODELING.md)** -- Train, evaluate, and track models using the SDK (16-cell notebook walkthrough including visualizations and dashboards)
3. **[Visualization Guide](docs/VISUALIZATIONS.md)** -- All 9 backends, `render()` function, dashboards, and the in-browser editor (pre-loaded as `visualization.ipynb` in workspaces)
4. **[Registry & CLI Guide](docs/CLI-REGISTRY.md)** -- Install, use, and manage models from the Open Model Registry (pre-loaded as `registry.ipynb` in workspaces)

---

Expand Down Expand Up @@ -222,6 +287,38 @@ Follow these guides to go from zero to a fully tracked ML experiment:
| `GET` | `/training/:id` | Get training job status |
| `GET` | `/training/:id/metrics` | SSE stream of training metrics |

### Visualizations & Dashboards

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/visualizations` | List visualizations (supports `?project_id=`) |
| `POST` | `/visualizations` | Create a visualization |
| `GET` | `/visualizations/:id` | Get visualization details |
| `PUT` | `/visualizations/:id` | Update visualization code/config |
| `POST` | `/visualizations/:id/render` | Render a visualization |
| `POST` | `/visualizations/:id/publish` | Publish for dashboard use |
| `GET` | `/dashboards` | List dashboards |
| `POST` | `/dashboards` | Create a dashboard |
| `PUT` | `/dashboards/:id` | Update dashboard layout |

### Notifications & Search

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/notifications` | Get user notifications (supports `?unread=true`) |
| `POST` | `/notifications/:id/read` | Mark notification as read |
| `POST` | `/notifications/read-all` | Mark all notifications as read |
| `GET` | `/search?q=` | Global search across models, datasets, experiments, jobs, projects |

### Model Registry

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/models/registry-status?names=` | Check install status for registry models |
| `POST` | `/models/registry-install` | Register a model from the registry |
| `POST` | `/models/registry-uninstall` | Unregister a registry model |
| `GET` | `/sdk/models/resolve-registry/:name` | Resolve a registry model by name (used by SDK `use_model()`) |

### Other Endpoints

| Method | Endpoint | Description |
Expand Down Expand Up @@ -304,7 +401,9 @@ Run `make help` to see all available targets. Key ones:
| Doc | Description |
|-----|-------------|
| [Usage Guide](docs/USAGE.md) | UI walkthrough: login, projects, datasets, workspaces |
| [Modeling Guide](docs/MODELING.md) | End-to-end SDK notebook: train, evaluate, track |
| [Modeling Guide](docs/MODELING.md) | End-to-end SDK notebook: train, evaluate, visualize, track |
| [Visualizations Guide](docs/VISUALIZATIONS.md) | 9 backends, `render()`, dashboards, in-browser editor |
| [CLI & Registry Guide](docs/CLI-REGISTRY.md) | Model registry: search, install, `use_model()`, uninstall |
| [Architecture](docs/ARCHITECTURE.md) | System design, component diagram, data flow |
| [Model Authoring](docs/MODEL-AUTHORING.md) | How to write models for OpenModelStudio |
| [Dataset Guide](docs/DATASET-GUIDE.md) | Preparing and uploading datasets |
Expand Down
30 changes: 30 additions & 0 deletions api/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ async fn main() {
.route("/projects/{project_id}/models", get(routes::models::list))
.route("/models", get(routes::models::list_all))
.route("/models", post(routes::models::create))
.route("/models/registry-status", get(routes::models::registry_status))
.route("/models/registry-uninstall", post(routes::models::registry_uninstall))
.route("/models/{id}", get(routes::models::get))
.route("/models/{id}", put(routes::models::update))
.route("/models/{id}", delete(routes::models::delete))
Expand Down Expand Up @@ -152,7 +154,9 @@ async fn main() {
.route("/features/{id}", delete(routes::features::delete))
// Notifications
.route("/notifications", get(routes::notifications::list))
.route("/notifications/unread-count", get(routes::notifications::unread_count))
.route("/notifications/read", post(routes::notifications::mark_read))
.route("/notifications/read-all", post(routes::notifications::mark_all_read))
// Search
.route("/search", get(routes::search::search))
// LLM
Expand All @@ -178,6 +182,7 @@ async fn main() {
.route("/sdk/datasets/{id}/upload", post(routes::sdk::dataset_upload))
.route("/sdk/datasets/{id}/content", get(routes::sdk::dataset_content))
.route("/sdk/models/resolve/{name_or_id}", get(routes::sdk::resolve_model))
.route("/sdk/models/resolve-registry/{name}", get(routes::sdk::resolve_registry_model))
.route("/sdk/models/{id}/artifact", get(routes::sdk::model_artifact))
// SDK Feature Store
.route("/sdk/features", post(routes::sdk::create_features))
Expand All @@ -201,6 +206,31 @@ async fn main() {
.route("/sdk/sweeps", post(routes::sdk::create_sweep))
.route("/sdk/sweeps/{id}", get(routes::sdk::get_sweep))
.route("/sdk/sweeps/{id}/stop", post(routes::sdk::stop_sweep))
// SDK Visualizations
.route("/sdk/visualizations", get(routes::visualizations::list_all))
.route("/sdk/visualizations", post(routes::visualizations::create))
.route("/sdk/visualizations/{id}", get(routes::visualizations::get))
.route("/sdk/visualizations/{id}", put(routes::visualizations::update))
.route("/sdk/visualizations/{id}/publish", post(routes::visualizations::publish))
.route("/sdk/visualizations/{id}/render", post(routes::visualizations::get))
// SDK Dashboards
.route("/sdk/dashboards", get(routes::visualizations::list_dashboards))
.route("/sdk/dashboards", post(routes::visualizations::create_dashboard))
.route("/sdk/dashboards/{id}", get(routes::visualizations::get_dashboard))
.route("/sdk/dashboards/{id}", put(routes::visualizations::update_dashboard))
// Visualizations
.route("/visualizations", get(routes::visualizations::list_all))
.route("/visualizations", post(routes::visualizations::create))
.route("/visualizations/{id}", get(routes::visualizations::get))
.route("/visualizations/{id}", put(routes::visualizations::update))
.route("/visualizations/{id}", delete(routes::visualizations::delete))
.route("/visualizations/{id}/publish", post(routes::visualizations::publish))
// Dashboards
.route("/dashboards", get(routes::visualizations::list_dashboards))
.route("/dashboards", post(routes::visualizations::create_dashboard))
.route("/dashboards/{id}", get(routes::visualizations::get_dashboard))
.route("/dashboards/{id}", put(routes::visualizations::update_dashboard))
.route("/dashboards/{id}", delete(routes::visualizations::delete_dashboard))
// Admin
.route("/admin/users", get(routes::admin::list_users))
.route("/admin/users/{id}", put(routes::admin::update_user))
Expand Down
4 changes: 2 additions & 2 deletions api/src/models/dataset.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use uuid::Uuid;
#[derive(Debug, Clone, Serialize, Deserialize, FromRow)]
pub struct Dataset {
pub id: Uuid,
pub project_id: Uuid,
pub project_id: Option<Uuid>,
pub name: String,
pub description: Option<String>,
pub format: String,
Expand Down Expand Up @@ -45,7 +45,7 @@ pub struct UploadUrlResponse {
#[derive(Debug, Clone, Serialize, Deserialize, FromRow)]
pub struct DataSource {
pub id: Uuid,
pub project_id: Uuid,
pub project_id: Option<Uuid>,
pub name: String,
pub source_type: String,
pub connection_string: Option<String>,
Expand Down
3 changes: 2 additions & 1 deletion api/src/models/model.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use uuid::Uuid;
#[derive(Debug, Clone, Serialize, Deserialize, FromRow)]
pub struct Model {
pub id: Uuid,
pub project_id: Uuid,
pub project_id: Option<Uuid>,
pub name: String,
pub description: Option<String>,
pub framework: String,
Expand All @@ -18,6 +18,7 @@ pub struct Model {
pub status: String,
pub language: String,
pub origin_workspace_id: Option<Uuid>,
pub registry_name: Option<String>,
}

#[derive(Debug, Deserialize)]
Expand Down
2 changes: 1 addition & 1 deletion api/src/models/pipeline.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use uuid::Uuid;
#[derive(Debug, Clone, Serialize, Deserialize, FromRow)]
pub struct Pipeline {
pub id: Uuid,
pub project_id: Uuid,
pub project_id: Option<Uuid>,
pub name: String,
pub description: Option<String>,
pub config: serde_json::Value,
Expand Down
51 changes: 37 additions & 14 deletions api/src/routes/automl.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use axum::{
extract::State,
extract::{Query, State},
Json,
};

Expand All @@ -11,26 +11,49 @@ use crate::AppState;
pub async fn list_sweeps(
State(state): State<AppState>,
AuthUser(_claims): AuthUser,
Query(params): Query<super::ProjectFilter>,
) -> AppResult<Json<Vec<Experiment>>> {
let sweeps: Vec<Experiment> = sqlx::query_as(
"SELECT * FROM experiments WHERE experiment_type = 'automl' ORDER BY created_at DESC"
)
.fetch_all(&state.db)
.await?;
let sweeps: Vec<Experiment> = if let Some(pid) = params.project_id {
sqlx::query_as(
"SELECT * FROM experiments WHERE experiment_type = 'automl' AND project_id = $1 ORDER BY created_at DESC"
)
.bind(pid)
.fetch_all(&state.db)
.await?
} else {
sqlx::query_as(
"SELECT * FROM experiments WHERE experiment_type = 'automl' ORDER BY created_at DESC"
)
.fetch_all(&state.db)
.await?
};
Ok(Json(sweeps))
}

pub async fn list_trials(
State(state): State<AppState>,
AuthUser(_claims): AuthUser,
Query(params): Query<super::ProjectFilter>,
) -> AppResult<Json<Vec<ExperimentRun>>> {
let trials: Vec<ExperimentRun> = sqlx::query_as(
"SELECT er.* FROM experiment_runs er
JOIN experiments e ON er.experiment_id = e.id
WHERE e.experiment_type = 'automl'
ORDER BY er.created_at DESC"
)
.fetch_all(&state.db)
.await?;
let trials: Vec<ExperimentRun> = if let Some(pid) = params.project_id {
sqlx::query_as(
"SELECT er.* FROM experiment_runs er
JOIN experiments e ON er.experiment_id = e.id
WHERE e.experiment_type = 'automl' AND e.project_id = $1
ORDER BY er.created_at DESC"
)
.bind(pid)
.fetch_all(&state.db)
.await?
} else {
sqlx::query_as(
"SELECT er.* FROM experiment_runs er
JOIN experiments e ON er.experiment_id = e.id
WHERE e.experiment_type = 'automl'
ORDER BY er.created_at DESC"
)
.fetch_all(&state.db)
.await?
};
Ok(Json(trials))
}
18 changes: 12 additions & 6 deletions api/src/routes/data_sources.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use axum::{
extract::{Path, State},
extract::{Path, Query, State},
Json,
};
use uuid::Uuid;
Expand All @@ -26,12 +26,18 @@ pub async fn list(
pub async fn list_all(
State(state): State<AppState>,
AuthUser(_claims): AuthUser,
Query(params): Query<super::ProjectFilter>,
) -> AppResult<Json<Vec<DataSource>>> {
let sources: Vec<DataSource> = sqlx::query_as(
"SELECT * FROM data_sources ORDER BY created_at DESC"
)
.fetch_all(&state.db)
.await?;
let sources: Vec<DataSource> = if let Some(pid) = params.project_id {
sqlx::query_as("SELECT * FROM data_sources WHERE project_id = $1 ORDER BY created_at DESC")
.bind(pid)
.fetch_all(&state.db)
.await?
} else {
sqlx::query_as("SELECT * FROM data_sources ORDER BY created_at DESC")
.fetch_all(&state.db)
.await?
};
Ok(Json(sources))
}

Expand Down
Loading
Loading