docs(refactor): complete admin service migration
This commit is contained in:
@@ -12,7 +12,7 @@ Multi-service monorepo for the Darano financial/crypto platform. Each subdirecto
|
||||
| `auth/` | Go (gRPC) | Authorization — OTP, JWT, permissions, identity |
|
||||
| `wallet/` | Go (gRPC) | Wallet — assets, transactions, Stellar blockchain, market |
|
||||
| `ui/` | TypeScript/Next.js | Customer-facing frontend |
|
||||
| `AdminPanel/` | Python/Django | Internal admin panel (bypasses api, hits Postgres directly) |
|
||||
| `AdminPanel/` | Python/Django | Internal admin: direct SQL reads, authenticated service-owned writes |
|
||||
| `proto/` | Protobuf | Central schema definitions shared by all services |
|
||||
| `DevOps/` | Docker Compose | Infrastructure — Postgres, Redis, RabbitMQ, MinIO, Traefik |
|
||||
| `docs/` | MkDocs | Documentation site |
|
||||
@@ -23,7 +23,8 @@ Multi-service monorepo for the Darano financial/crypto platform. Each subdirecto
|
||||
|
||||
```
|
||||
Browser/Client → api (REST/HTTP) → auth, wallet/market/alert (gRPC)
|
||||
AdminPanel ──────────────────────→ Postgres directly (bypasses api)
|
||||
AdminPanel ──reads───────────────→ Postgres
|
||||
AdminPanel ──authenticated gRPC──→ wallet (asset administration)
|
||||
```
|
||||
|
||||
The `api` gateway is the **only** HTTP-facing service. All inter-service communication is gRPC.
|
||||
@@ -198,15 +199,14 @@ All Go services instrumented with Elastic APM and Prometheus metrics. Traefik ha
|
||||
|
||||
## AdminPanel Deep Dive
|
||||
|
||||
- **Architecture**: Django admin panel that bypasses all Go backend services. Connects directly to `core_db` (same Postgres as Go services) for read/write. Uses Django DB routers in `src/adminpanel/db/routers.py` to route `coreLogic` models to `core_db` and other apps to `default`.
|
||||
- **Architecture**: Django admin panel that reads `core_db` directly for fast projections. Unmanaged models are read-only by default; the Assets admin sends authenticated typed commands to Wallet instead of writing the database. Django DB routers route `coreLogic` reads to `core_db` and other apps to `default`.
|
||||
- **Models**: `src/coreLogic/models.py` contains `managed = False` Django models — manual replicas of Go GORM models generated via `inspectdb`. **Not auto-generated from protos.** `make proto` generates betterproto Python stubs separately but Django models are maintained by hand. **Never edit models.py manually** — it drifts from Go services.
|
||||
- **Admin classes**: `src/coreLogic/admin/` — 17 admin files (`asset.py`, `wallets.py`, `market.py`, etc.). All inherit from `MultiDBModelAdmin` in `src/utils/base_admin.py` which handles: multi-database writes (`using="core_db"`), soft deletes via `deleted_at`, asset-level permission filtering, and Jalali date widgets.
|
||||
- **Permission system**: `src/usermapper/user_perm.py` + `src/coreLogic/acl.py` — admin users get asset-level access control. `save_model` checks `user_perm.can_access_asset()` before allowing writes.
|
||||
- **Key gotchas**:
|
||||
- `check_token_policy()` in `admin/asset.py:161` duplicates validation from wallet service. Changes to asset validation must be made in **both** places.
|
||||
- `GENERIC_ASSET_META_VALUE` in `admin/asset.py:31` is a hardcoded JSON blob — if the wallet service changes asset metadata structure, this must be updated too.
|
||||
- The router has a typo: `no_migartion` → should be `no_migration`.
|
||||
- All `coreLogic` models are read via `core_db` directly. **Any admin write bypasses Go services** — no trustline updates, no blockchain operations, no validation from wallet/market services.
|
||||
- Asset policy and default metadata are owned by Wallet's `application/adminasset`; do not recreate them in Django.
|
||||
- The Wallet admin client requires matching AdminPanel `WALLET_ADMIN_GRPC_TOKEN` and internal-wallet `[admin-assets].token` configuration.
|
||||
- All `coreLogic` models are read via `core_db` directly. Adding a mutation requires an explicit typed owning-service workflow; never opt an unmanaged model into direct ORM writes.
|
||||
|
||||
## Ongoing Refactoring: DDD / Clean Architecture
|
||||
|
||||
@@ -214,7 +214,7 @@ This repository is being migrated to a unified Domain-Driven Design / Clean Arch
|
||||
|
||||
**Go services current state**: API, Auth, and Wallet configuration and architecture phases are complete on `feat/refactor-v1`; use the refactoring tracker and audit for the current package boundaries and verification evidence.
|
||||
|
||||
**AdminPanel current state**: Direct DB access to `core_db` with no gRPC layer. Business logic duplicated in Django admin classes (`check_token_policy`, `auto_gen` instead of delegating to Go services). `managed = False` models drift from Go GORM models.
|
||||
**AdminPanel current state**: Direct DB reads remain, while unmanaged projections fail closed for writes. Asset upsert/deactivation uses an authenticated Wallet gRPC adapter; duplicated asset policy and metadata generation have been removed from Django. `managed = False` models can still drift from Go persistence models and must remain projection-only.
|
||||
|
||||
**Target**: All Go services follow `domain/` → `application/` → `infrastructure/` → `interface/` with inward dependencies. AdminPanel routes writes through gRPC to Go services while keeping reads from DB for performance.
|
||||
|
||||
|
||||
@@ -103,6 +103,10 @@ Final Wallet verification on 2026-08-31: protobuf regeneration produced no diff;
|
||||
|
||||
No API refactoring task remains open. Final API verification on 2026-08-31: protobuf and Swagger regeneration produced no diff; `go test ./...`, `go test -race ./...`, `go vet ./...`, and `go build ./...` all passed. Swagger regeneration requires `GOCACHE=/tmp/darano-api-go-cache` in this environment.
|
||||
|
||||
### AdminPanel/protobuf completion state
|
||||
|
||||
`P001`–`P010` and `R001`–`R003` are complete. AdminPanel still reads service-owned tables directly, but every unmanaged model is read-only unless explicitly service-backed. Assets write through authenticated Wallet gRPC upsert/deactivate commands; prices and every other legacy mutation/import/inline/bulk path are disabled. Configure matching `WALLET_ADMIN_GRPC_TOKEN` in AdminPanel and `[admin-assets].token` in the internal-wallet configuration, plus the optional `WALLET_ADMIN_GRPC_ADDRESS` and `WALLET_ADMIN_GRPC_TIMEOUT` values. AdminPanel protobuf generation now uses the adjacent local `proto/` checkout and BetterProto 2.
|
||||
|
||||
### Commands and cautions
|
||||
|
||||
- Prefix shell commands with `rtk` as required by `/home/navid/.codex/RTK.md`.
|
||||
|
||||
@@ -520,3 +520,12 @@ Migration order implied by the map: isolate shared infrastructure constructors (
|
||||
- `G007`: `cmd/apiRuntime` is the explicit dependency graph for upstream clients, HTTP handlers, router, server ownership, startup permission synchronization, and shutdown. Construction is side-effect-free and its complete 61-route graph is tested.
|
||||
- `G008`: removed root `service`, `handler`, and `middlewares` packages; the generated service enum/reflection lookup, unused mock-data helper, empty error-enum shell, commented legacy handlers/routes, and `gofakeit` dependency are gone. No legacy package import or duplicate client/handler implementation remains.
|
||||
- Final API commit is `31e3043` on `feat/refactor-v1`. `make build-proto` and writable-cache `make build-swag` reproduce committed outputs with no diff; `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`, and whitespace checks pass.
|
||||
|
||||
## AdminPanel/protobuf phase completion — `P001`–`P010`, `R001`–`R003` (2026-08-31)
|
||||
|
||||
- `AdminPanel/ADMIN-MUTATION-INVENTORY.md` is the evidence baseline. Asset administration was the only sufficiently specified missing service contract; generic wallet, transaction, market, auth, or configuration CRUD was rejected because it would bypass aggregate invariants.
|
||||
- Proto adds only typed, additive `AdminUpsertAsset` and `AdminDeactivateAsset` commands. Wallet authenticates them using fail-closed `x-admin-token` metadata and configurable token state.
|
||||
- `application/adminasset` owns activation constraints, decimal and buy-limit validation, metadata defaults, token-type derivation, persistence, cache invalidation, and safe deactivation. Custody fields absent from the legacy Django projection are preserved on update.
|
||||
- AdminPanel generates its active BetterProto message subset from the adjacent local proto checkout. Its wallet adapter applies configurable address, timeout, and credential metadata and translates permission, validation, timeout, transport, and status failures.
|
||||
- Unmanaged Django models are read-only by default. Assets are the sole explicit service-backed exception; asset prices, inlines, imports, bulk deletion, legacy persistence signals, and all other direct mutation paths are disabled.
|
||||
- Router typo and tests are complete. Buf lint/build/breaking checks pass; full Wallet tests pass; thirteen AdminPanel boundary tests, Django checks, compilation, regeneration, and whitespace checks pass.
|
||||
|
||||
+16
-13
@@ -149,24 +149,24 @@ This is the authoritative execution tracker for the refactor. Work is performed
|
||||
|
||||
| ID | Status | Task | Acceptance check / note |
|
||||
|---|---|---|---|
|
||||
| P001 | `TODO` | Inventory every direct AdminPanel write, delete, inline, bulk action, validation, and side effect. | Each mutation is mapped to an owning Go service or explicitly classified. |
|
||||
| P002 | `TODO` | Compare required AdminPanel mutations with existing internal RPCs. | Every missing contract is documented before any proto change. |
|
||||
| P003 | `TODO` | Add reusable authenticated, deadline-aware gRPC client infrastructure. | Unit tests cover success, timeout, unavailable service, and status translation. |
|
||||
| P004 | `TODO` | Add asset application use cases and route asset writes through wallet RPCs. | Direct asset writes stop; access checks and user-visible errors remain correct. |
|
||||
| P005 | `TODO` | Move token-policy validation and metadata generation to Go-owned operations. | Django no longer owns duplicate wallet business rules. |
|
||||
| P006 | `TODO` | Add wallet and transaction use cases and migrate writes. | Direct writes stop and failure/partial-write behavior is tested. |
|
||||
| P007 | `TODO` | Add market and remaining mutable aggregate use cases. | All classified mutations use their owning service. |
|
||||
| P008 | `TODO` | Make legacy unmanaged ORM models explicitly read-only. | List/detail reads remain fast; migrated writes cannot bypass services. |
|
||||
| P009 | `TODO` | Fix `no_migartion` to `no_migration` with router tests. | Correct router setting is covered without unintended migrations. |
|
||||
| P010 | `TODO` | Verify every affected admin page and permission tier. | Create/update/delete/inline/bulk/failure paths pass. |
|
||||
| P001 | `DONE` | Inventory every direct AdminPanel write, delete, inline, bulk action, validation, and side effect. | `ADMIN-MUTATION-INVENTORY.md` maps every mutation surface and side effect to its owner or a read-only decision. |
|
||||
| P002 | `DONE` | Compare required AdminPanel mutations with existing internal RPCs. | The inventory proves asset administration is the only sufficiently specified missing contract. |
|
||||
| P003 | `DONE` | Add reusable authenticated, deadline-aware gRPC client infrastructure. | Typed BetterProto/grpclib adapter uses token metadata and configurable deadlines; tests cover success, timeout, unavailable, permission, validation, and status handling. |
|
||||
| P004 | `DONE` | Add asset application use cases and route asset writes through wallet RPCs. | Wallet owns upsert/deactivate; AdminPanel never saves or deletes an asset through its ORM, preserves asset access checks, and surfaces RPC failures. |
|
||||
| P005 | `DONE` | Move token-policy validation and metadata generation to Go-owned operations. | Wallet validates activation/amount policy and derives metadata/token type; duplicate Django logic and related-row auto-generation were removed. |
|
||||
| P006 | `DONE` | Add wallet and transaction use cases and migrate writes. | Inventory found no valid administrative CRUD: existing Wallet workflows remain the only mutation path and wallet/transaction projections are strictly read-only. |
|
||||
| P007 | `DONE` | Add market and remaining mutable aggregate use cases. | Existing lifecycle RPCs remain authoritative; speculative CRUD was rejected and all remaining projections/imports/inlines are read-only. |
|
||||
| P008 | `DONE` | Make legacy unmanaged ORM models explicitly read-only. | Base admin fails closed for unmanaged models; save/delete/bulk/inline/import bypasses are disabled, with a registry-wide regression test. |
|
||||
| P009 | `DONE` | Fix `no_migartion` to `no_migration` with router tests. | Router tests prove `coreLogic` never migrates on either database and managed apps remain allowed. |
|
||||
| P010 | `DONE` | Verify every affected admin page and permission tier. | Thirteen focused tests cover registry-wide permissions, asset create/update/deactivate, bulk rejection, service failure, client statuses, and router behavior; Django checks pass. |
|
||||
|
||||
## Phase 6 — Protobuf contracts
|
||||
|
||||
| ID | Status | Task | Acceptance check / note |
|
||||
|---|---|---|---|
|
||||
| R001 | `TODO` | Decide whether existing internal RPCs cover all required AdminPanel writes. | Decision is evidence-based; no speculative contract changes. |
|
||||
| R002 | `TODO` | Add backward-compatible internal RPCs only for proven gaps. | Buf lint/breaking checks pass; existing field numbers and contracts are preserved. |
|
||||
| R003 | `TODO` | Regenerate only active Go and Python consumers. | Generated files are reproducible and Go JSON tags retain required field presence. |
|
||||
| R001 | `DONE` | Decide whether existing internal RPCs cover all required AdminPanel writes. | Evidence in `ADMIN-MUTATION-INVENTORY.md` limits new contracts to typed asset administration and rejects generic table mutation. |
|
||||
| R002 | `DONE` | Add backward-compatible internal RPCs only for proven gaps. | Additive upsert/deactivate messages and methods pass Buf lint, build, and breaking checks against the prior commit. |
|
||||
| R003 | `DONE` | Regenerate only active Go and Python consumers. | Wallet Go stubs and the AdminPanel's local typed message subset are reproducibly generated; required Go JSON field presence is retained. |
|
||||
|
||||
## Phase 7 — Shared types and final verification
|
||||
|
||||
@@ -348,3 +348,6 @@ Append one row whenever a task changes status. Existing rows are never rewritten
|
||||
| 2026-08-31 | G006 | `TODO` | `DONE` | Extracted router, health, metrics, Swagger, WebSocket, HTTP/pprof server ownership, and graceful shutdown into `interface/http` in API `380aad4`. |
|
||||
| 2026-08-31 | G007 | `TODO` | `DONE` | Added explicit typed API runtime composition with startup permission synchronization and owned cleanup in API `163114f`. |
|
||||
| 2026-08-31 | G008 | `TODO` | `DONE` | Removed legacy service/handler/middleware packages, generated enum/mock shims, dead shells, and unused dependencies in API `31e3043`; protobuf/Swagger regeneration, tests, race, vet, build, and whitespace checks pass. |
|
||||
| 2026-08-31 | P001–P002 / R001 | `TODO` | `DONE` | Audited every AdminPanel mutation surface and existing internal RPC, documenting why asset administration is the sole proven contract gap and why arbitrary wallet, transaction, market, auth, and configuration CRUD must remain disabled. |
|
||||
| 2026-08-31 | R002–R003 | `TODO` | `DONE` | Added additive authenticated asset upsert/deactivate contracts, passed Buf lint/build/breaking checks, regenerated Wallet Go stubs, and made the AdminPanel Python subset locally reproducible. |
|
||||
| 2026-08-31 | P003–P010 | `TODO` | `DONE` | Added the deadline/token-aware Wallet client; moved asset policy and persistence into Wallet; made every other unmanaged admin, inline, import, delete, and signal path fail closed; fixed the migration router; full Wallet tests plus thirteen AdminPanel boundary tests and Django checks pass. |
|
||||
|
||||
Reference in New Issue
Block a user