docs(refactor): complete final verification and handoff

This commit is contained in:
2026-09-01 14:34:12 +03:30
parent 7d2008772e
commit a504dc2b03
6 changed files with 222 additions and 15 deletions
+11 -10
View File
@@ -48,7 +48,7 @@ Single binary hosts multiple sub-services via Cobra subcommands: `wallet`, `mark
### Config
- **Go services** use `knadh/koanf` (not `fig` as older docs may suggest) to parse TOML config files
- **Global singleton**: `config.Cfg` — a `sync.Once` ensures it's initialized exactly once
- **Explicit ownership**: commands load configuration once and inject it through composition; legacy `config.Cfg` globals were removed
- **Defaults baked into code**: `config.go` in `api/` sets default `ipg_callback_url` and `ui_server_error_status_url` before loading the TOML file (TOML overrides defaults)
- **Struct tags**: use `koanf:"field-name"` (not env vars)
@@ -133,18 +133,19 @@ Proto definitions live in `proto/` with subdirectories: `base/`, `auth/`, `walle
## Go Service Layer Pattern
Both `auth` and `wallet` follow a layered architecture:
The active Go services follow inward-facing layers:
- **`config/`** — TOML config via koanf. `config.Cfg` global singleton, `sync.Once` initialized.
- **`domain/`** — entities, exact value objects, errors, and ports without transport/framework ownership.
- **`application/`** — business policies and orchestration against domain ports.
- **`infrastructure/`** — koanf configuration, PostgreSQL/Redis, external clients, queues, Stellar, and generated-client adapters.
- **`interface/`** — gRPC/HTTP/process adapters and protocol mapping.
- **`domain/stub/go/`** — Generated protobuf Go code. **Never edit manually.**
- **`cmd/`** — Cobra CLI entry points. Subcommands map to serve modes.
- **`repository/`** — Data access. Aggregates `IPostgres`, `IRedis`, `IService`, and `IQueue` (wallet only) interfaces.
- **`core/`** (wallet) / **`usecase/`** (auth) — Business logic. Implements gRPC server interfaces from proto.
- **`util/`** — Shared helpers; no business logic.
- **`repository/` / `usecase/`** — remaining compatibility composition/interfaces; implementations live in infrastructure and new business logic belongs in application/domain.
In `auth`, `usecase.UseCase` interface directly embeds the generated gRPC server interfaces (`authv1.AuthorizationServiceServer`, `authv1.InternalAuthorizationServiceServer`).
In `wallet`, `core/` contains sub-packages: `walletImp/`, `marketImp/`, `alertImp/`, `cronJobs/`.
In `wallet`, runtime adapters live under `interface/grpc` and `interface/process`; the superseded `core/*Imp` packages no longer exist.
## API Gateway Patterns
@@ -195,14 +196,14 @@ All Go services instrumented with Elastic APM and Prometheus metrics. Traefik ha
7. **Proto stubs excluded from watchers**: `.air.toml` excludes `domain/stub/` from file watching. Air also excludes `swagger/` and `testdata/`.
8. **API testing requests**: `api/req/` contains JS files for API request testing (axios-based) — not part of the app, useful for manual testing.
9. **Django Admin**: Uses `unfold` theme (AdminPanel admin customizations in `sites.py` and `unfoldconf.py`).
10. **Proto generation for AdminPanel**: Generates into root directories (`base/`, `wallet/`, etc.) then deletes them with `rm -rf`. The betterproto generator outputs Python classes directly.
10. **Proto generation for AdminPanel**: `make proto` uses the adjacent local `proto/` checkout and generates only the active Base/Auth/Wallet BetterProto message subset into `src/stub/`.
## AdminPanel Deep Dive
- **Architecture**: Django admin panel that reads `core_db` directly for fast projections. Unmanaged models are read-only by default; the Assets admin sends authenticated typed commands to Wallet instead of writing the database. Django DB routers route `coreLogic` reads to `core_db` and other apps to `default`.
- **Models**: `src/coreLogic/models.py` contains `managed = False` Django models — manual replicas of Go GORM models generated via `inspectdb`. **Not auto-generated from protos.** `make proto` generates betterproto Python stubs separately but Django models are maintained by hand. **Never edit models.py manually** — it drifts from Go services.
- **Admin classes**: `src/coreLogic/admin/` 17 admin files (`asset.py`, `wallets.py`, `market.py`, etc.). All inherit from `MultiDBModelAdmin` in `src/utils/base_admin.py` which handles: multi-database writes (`using="core_db"`), soft deletes via `deleted_at`, asset-level permission filtering, and Jalali date widgets.
- **Permission system**: `src/usermapper/user_perm.py` + `src/coreLogic/acl.py` — admin users get asset-level access control. `save_model` checks `user_perm.can_access_asset()` before allowing writes.
- **Admin classes**: `src/coreLogic/admin/` — admin files inherit from `MultiDBModelAdmin` in `src/utils/base_admin.py`, which routes reads to `core_db`, applies asset-level filters/Jalali widgets, and makes unmanaged projections fail closed for add/change/delete. Assets opt into service-backed mutations only.
- **Permission system**: `src/usermapper/user_perm.py` + `src/coreLogic/acl.py` — admin users get asset-level access control. Asset service commands enforce the same access decision before dispatch.
- **Key gotchas**:
- Asset policy and default metadata are owned by Wallet's `application/adminasset`; do not recreate them in Django.
- The Wallet admin client requires matching AdminPanel `WALLET_ADMIN_GRPC_TOKEN` and internal-wallet `[admin-assets].token` configuration.
+108
View File
@@ -0,0 +1,108 @@
# Darano refactoring final migration report
Date: 2026-09-01
Branch: `feat/refactor-v1`
Authoritative tracker: `REFACTORING-TODO.md`
## Outcome
The planned refactoring is complete. There are no `TODO`, `STARTED`, or `FAILED`
tasks in the actionable tracker. Historical `CHANGED` entries preserve explicit
scope decisions and completed task splits.
Active repositories are API, Auth, Wallet, AdminPanel, Proto, and GL. UI, Docs,
and DevOps were excluded from the architecture refactor by scope decision and were
not modified during the final phase.
## Delivered architecture
- API owns the public HTTP boundary under `interface/http`, depends on an
application upstream port, and uses explicitly composed lazy gRPC clients.
- Auth configuration is injected; domain/application boundaries cover OTP,
session/JWT, identity, periodic identity validation, and permissions; persistence
and provider implementations are infrastructure-owned.
- Wallet uses exact scale-7 domain money, application-owned wallet/transaction,
market, alert, stream, locking, and asset-administration workflows, explicit
process composition, durable transaction/ledger outboxes, and interface-owned
adapters.
- GL is an independent immutable double-entry service with scale-18 audit amounts,
idempotent append/reversal, reconciliation, recovery, explorer, and load coverage.
- Publisher-backed ICO settlement and all transaction-type event routes use the
owning Wallet/Market workflows with durable, idempotent delivery.
- Federation coupling and wallet creation by federation were removed; ownership is
`user_id -> identity_id -> wallet_id`, with wallets remaining asset-scoped.
- AdminPanel retains direct SQL reads for projections. Unmanaged projections fail
closed for writes; Assets alone use authenticated typed Wallet upsert/deactivate
commands. Duplicate Django wallet policy and secondary persistence signals are gone.
- Proto changes are additive where compatibility was required, and active generated
consumers are committed and reproducible.
## Shared-type decision
`SHARED-TYPES-EVALUATION.md` concludes that no new shared source package is safe:
- Wallet uses scale 7 with Stellar and `numeric(23,7)` semantics.
- GL uses precision 38/scale 18 with canonical immutable-journal semantics.
- deployed IDs mix signed database keys, unsigned legacy contract fields, Auth-owned
regulated identifiers, and opaque GL strings.
The approved boundary remains canonical base-10 decimal strings plus explicit
adapter conversions. Services do not import another service's domain package.
## Final corrections found by verification
- API commit `0074e91` regenerates its Wallet contract consumer.
- Auth commit `2c8a0b1` regenerates its Wallet contract consumer.
- Wallet commit `3b898e1` makes the duplicate-event integration assertion wait for
the first inbox success transition. The previous test could publish its duplicate
before the first handler completed; 20 race-enabled repetitions and the full race
suite pass after the correction.
These changes are scoped to generated outputs or test synchronization. No unrelated
working-tree changes were absorbed.
## Final verification
| Repository | Verification |
|---|---|
| Proto | Buf lint, build, and breaking check against the prior commit pass. |
| API | Proto and Swagger generation; `go test ./...`; race tests; vet; build; whitespace checks pass. |
| Auth | Proto generation; `go test ./...`; race tests; vet; build; whitespace checks pass. |
| Wallet | Proto generation; normal and race tests; vet; build; focused duplicate-event race test repeated 20 times; whitespace checks pass. |
| GL | Templ/proto generation; normal and race tests; vet; build pass. The localhost `httptest` case required normal socket permission rather than the restricted sandbox. |
| AdminPanel | Local BetterProto generation; 13 focused boundary tests; Django system checks; Python compilation; whitespace checks pass. |
API/Auth/Wallet/GL/AdminPanel/Proto are clean on `feat/refactor-v1` after their final
commits. The coordination repository contains only this final documentation change
until its handoff commit is created.
## Required deployment configuration
Asset administration is fail-closed. Configure matching secrets:
- AdminPanel: `WALLET_ADMIN_GRPC_TOKEN`
- Internal Wallet: `[admin-assets].token`
Optional AdminPanel settings:
- `WALLET_ADMIN_GRPC_ADDRESS` (default `127.0.0.1:8500`)
- `WALLET_ADMIN_GRPC_TIMEOUT` (default 5 seconds)
The internal Wallet schedule/config files remain environment-owned and ignored where
previously established; deploy secrets through the existing configuration mechanism.
## Known external/baseline conditions
- The Darano Go proxy previously returned long `504` responses during cold Docker
dependency downloads for API/Auth/Wallet. Native generation, tests, vet, and builds
pass; the final phase did not alter registry policy or add a public fallback.
- Django deploy checks still depend on production-provided secret, TLS redirect,
secure cookie, CSRF, HSTS, and DEBUG settings. Normal system checks pass.
- Legacy public Wallet/Market protobuf money remains `double` for compatibility.
New exact internal financial boundaries must use canonical decimal strings.
## Handoff
Use `REFACTORING-TODO.md`, `REFACTORING-AUDIT.md`, this report, and `MEMORY.md` as
the continuation sources. All implementation tasks are complete; future work should
be opened as a new scoped task rather than reopening the finished migration tracker.
+6
View File
@@ -107,6 +107,12 @@ No API refactoring task remains open. Final API verification on 2026-08-31: prot
`P001``P010` and `R001``R003` are complete. AdminPanel still reads service-owned tables directly, but every unmanaged model is read-only unless explicitly service-backed. Assets write through authenticated Wallet gRPC upsert/deactivate commands; prices and every other legacy mutation/import/inline/bulk path are disabled. Configure matching `WALLET_ADMIN_GRPC_TOKEN` in AdminPanel and `[admin-assets].token` in the internal-wallet configuration, plus the optional `WALLET_ADMIN_GRPC_ADDRESS` and `WALLET_ADMIN_GRPC_TIMEOUT` values. AdminPanel protobuf generation now uses the adjacent local `proto/` checkout and BetterProto 2.
### Final refactoring completion state (2026-09-01)
`F001``F005` are complete and the authoritative tracker has no actionable remaining task. `SHARED-TYPES-EVALUATION.md` records why Wallet scale-7 money, GL scale-18 money, legacy numeric IDs, Auth identifiers, and opaque GL IDs remain bounded-context types. `FINAL-MIGRATION-REPORT.md` records the final architecture, verification matrix, deployment configuration, known external conditions, and handoff.
Final verification regenerated every active consumer and found two missing checked-in consumers plus one race-test synchronization issue. API `0074e91`, Auth `2c8a0b1`, and Wallet `3b898e1` contain those corrections. Proto lint/build/breaking, all active Go normal/race/vet/build gates, Wallet's focused 20-run race test, AdminPanel's 13 focused tests/checks/compilation, and reproducible generation pass.
### Commands and cautions
- Prefix shell commands with `rtk` as required by `/home/navid/.codex/RTK.md`.
+8
View File
@@ -529,3 +529,11 @@ Migration order implied by the map: isolate shared infrastructure constructors (
- AdminPanel generates its active BetterProto message subset from the adjacent local proto checkout. Its wallet adapter applies configurable address, timeout, and credential metadata and translates permission, validation, timeout, transport, and status failures.
- Unmanaged Django models are read-only by default. Assets are the sole explicit service-backed exception; asset prices, inlines, imports, bulk deletion, legacy persistence signals, and all other direct mutation paths are disabled.
- Router typo and tests are complete. Buf lint/build/breaking checks pass; full Wallet tests pass; thirteen AdminPanel boundary tests, Django checks, compilation, regeneration, and whitespace checks pass.
## Final phase completion — `F001``F005` (2026-09-01)
- Cross-context evaluation rejected a universal ID or monetary source package: Wallet and GL precision/arithmetic invariants differ, while deployed identifier families have distinct ownership, signedness, namespace, and opacity. The evidence and approved conversion rules are in `SHARED-TYPES-EVALUATION.md`.
- Final regeneration corrected previously missing API/Auth Wallet admin-contract consumers (`0074e91`, `2c8a0b1`). Required JSON-tag behavior remains governed by each consumer's established generator workflow.
- Wallet's full race gate exposed a test synchronization window: the duplicate was published after notification but before the first inbox success transition. Commit `3b898e1` waits for the actual success transition; 20 focused race repetitions and the complete race suite pass.
- Every active Go repository passes generation, normal tests, race tests, vet, and native build. Proto passes lint/build/breaking. AdminPanel passes local generation, thirteen focused tests, Django checks, compilation, and whitespace checks.
- API/Auth/Wallet/GL/AdminPanel/Proto are clean on `feat/refactor-v1`; final results and deployment requirements are consolidated in `FINAL-MIGRATION-REPORT.md`.
+13 -5
View File
@@ -172,11 +172,11 @@ This is the authoritative execution tracker for the refactor. Work is performed
| ID | Status | Task | Acceptance check / note |
|---|---|---|---|
| F001 | `TODO` | Evaluate shared ID and monetary types after service boundaries stabilize. | Only genuinely identical cross-context semantics are shared. |
| F002 | `TODO` | Introduce approved shared types incrementally, if justified. | Versioned dependency and explicit boundary conversions are used. |
| F003 | `TODO` | Run final generation, tests, builds, and architecture checks. | Results meet or exceed the recorded baseline. |
| F004 | `TODO` | Review every active repository diff for generated/manual/unrelated changes. | Only scoped changes remain and user-owned work is preserved. |
| F005 | `TODO` | Prepare reviewable per-repository commits and final migration report. | Every commit has one purpose and leaves its repository buildable. |
| F001 | `DONE` | Evaluate shared ID and monetary types after service boundaries stabilize. | `SHARED-TYPES-EVALUATION.md` records the semantic matrix: no candidate has identical cross-context invariants. |
| F002 | `DONE` | Introduce approved shared types incrementally, if justified. | No shared source type is justified. Wallet scale-7 and GL scale-18 values remain local with canonical decimal-string conversion at their versioned boundary. |
| F003 | `DONE` | Run final generation, tests, builds, and architecture checks. | Proto, generated consumers, Django checks/tests, and all Go normal/race/vet/build gates pass; final generation gaps and one test synchronization defect were corrected. |
| F004 | `DONE` | Review every active repository diff for generated/manual/unrelated changes. | All six active code repositories are clean on `feat/refactor-v1`; regeneration diffs were scoped generated consumers, and the only manual verification diff was the race-test synchronization fix. |
| F005 | `DONE` | Prepare reviewable per-repository commits and final migration report. | Scoped API/Auth/Wallet commits are verified and `FINAL-MIGRATION-REPORT.md` records the complete outcome, gates, configuration, and handoff. |
## Execution log
@@ -351,3 +351,11 @@ Append one row whenever a task changes status. Existing rows are never rewritten
| 2026-08-31 | P001P002 / R001 | `TODO` | `DONE` | Audited every AdminPanel mutation surface and existing internal RPC, documenting why asset administration is the sole proven contract gap and why arbitrary wallet, transaction, market, auth, and configuration CRUD must remain disabled. |
| 2026-08-31 | R002R003 | `TODO` | `DONE` | Added additive authenticated asset upsert/deactivate contracts, passed Buf lint/build/breaking checks, regenerated Wallet Go stubs, and made the AdminPanel Python subset locally reproducible. |
| 2026-08-31 | P003P010 | `TODO` | `DONE` | Added the deadline/token-aware Wallet client; moved asset policy and persistence into Wallet; made every other unmanaged admin, inline, import, delete, and signal path fail closed; fixed the migration router; full Wallet tests plus thirteen AdminPanel boundary tests and Django checks pass. |
| 2026-09-01 | F001 | `TODO` | `STARTED` | Began a code-backed comparison of identifier and monetary semantics across Proto, Auth, Wallet, API, and GL before approving any shared representation. |
| 2026-09-01 | F001 | `STARTED` | `DONE` | Documented differing Wallet/GL precision, arithmetic, persistence, identifier ownership, signedness, and opaque-ID semantics in `SHARED-TYPES-EVALUATION.md`; no universal type is safe. |
| 2026-09-01 | F002 | `TODO` | `DONE` | Concluded that no new shared implementation is justified; retained independently versioned domain types and the existing lossless canonical decimal-string service boundary. |
| 2026-09-01 | F003 | `TODO` | `STARTED` | Began final reproducible generation, tests, race checks, vet, builds, Django checks, and architecture scans across every active repository. |
| 2026-09-01 | F003 | `STARTED` | `DONE` | Passed Buf lint/build/breaking, local Python and all Go generation, thirteen AdminPanel tests/checks, and API/Auth/Wallet/GL tests/race/vet/build. Committed missing API/Auth generated Wallet contracts and corrected Wallet integration-test synchronization found by the race gate. |
| 2026-09-01 | F004 | `TODO` | `STARTED` | Began final per-repository status, generated/manual diff, branch, architecture-documentation, and unrelated-change review across all active repositories. |
| 2026-09-01 | F004 | `STARTED` | `DONE` | Confirmed API/Auth/Wallet/GL/AdminPanel/Proto are clean on `feat/refactor-v1`; reviewed generated admin-contract diffs and the Wallet test-only synchronization diff; refreshed stale architecture guidance without modifying excluded repositories. |
| 2026-09-01 | F005 | `TODO` | `DONE` | Prepared purpose-specific generated-consumer and test commits plus `FINAL-MIGRATION-REPORT.md`; every active repository remains buildable and no actionable tracker task remains. |
+76
View File
@@ -0,0 +1,76 @@
# Shared ID and monetary type evaluation
Date: 2026-09-01
Scope: `api`, `auth`, `wallet`, `GL`, `proto`, and `AdminPanel`
Tasks: `F001`, `F002`
## Decision
No new cross-repository source package or universal protobuf wrapper is approved.
The superficially similar values do not currently have identical invariants. Keep
domain value objects local and perform explicit conversion at service boundaries.
This is a positive architectural decision, not deferred implementation. A shared
type may be proposed later only if its producer, consumers, precision, nullability,
validation, versioning, and compatibility behavior are proven identical.
## Monetary semantics
| Context | Representation | Invariants | Decision |
|---|---|---|---|
| Wallet domain | `domain/money.Amount`, fixed-scale integer units backed by `big.Int` | Scale 7; mirrors `numeric(23,7)` and Stellar stroops; supports exact wallet arithmetic and explicit legacy `float64` conversion | Keep Wallet-owned. |
| Wallet nullable persistence values | `money.NullAmount` | Distinguishes SQL null from zero | Keep Wallet-owned; nullability is persistence/domain-specific. |
| GL domain | `domain/ledger.Amount`, fixed-scale `big.Int` | Precision 38, scale 18; canonical parsing; immutable double-entry journal values | Keep GL-owned. It intentionally has more precision and stricter canonical input than Wallet. |
| Wallet ↔ GL contract | Canonical base-10 `string` in `ledger.v1.JournalEntry.amount` | Lossless, language-neutral boundary; Wallet formats its exact amount and GL reparses under GL constraints | Retain. This is the correct explicit conversion boundary. |
| Existing public Wallet/Market contracts | `double` plus some integer IRR/raw values | Backward-compatibility surface with explicit conversion/finite/range checks inside Wallet | Do not replace during this refactor; changing wire types is breaking. Track separately if a versioned v2 contract is approved. |
| AdminPanel | Python `Decimal`/Django decimal fields, serialized to canonical strings for asset administration | Projection/form representation; Wallet owns final validation | Keep adapter-local. |
Why Wallet and GL amounts must not be aliased:
- scale 7 and scale 18 represent different accepted value sets;
- their database precision limits differ;
- Wallet includes Stellar raw-unit conversion and commercial rounding operations;
- GL requires canonical journal serialization, conservation, and immutable audit precision;
- importing either implementation into the other repository would reverse the service dependency direction;
- a common implementation would still require context-specific wrappers, eliminating its claimed benefit.
## Identifier semantics
| Identifier family | Current representation | Meaning | Decision |
|---|---|---|---|
| Auth/User/Identity database IDs | Predominantly signed `int64`; national ID is validated `auth/model.NationalID` string | Auth-owned persistence identity versus a regulated external identifier | Keep `NationalID` Auth-owned. Do not conflate it with database IDs. |
| Wallet asset/user/transaction IDs | Predominantly signed `int64` | Positive database keys; zero commonly represents absent/default at protobuf boundaries | Keep explicit field names and validate positivity per operation. |
| Wallet market/order/legacy entity IDs | Mixture of `uint`, `uint64`, and `int64` | Historical GORM/protobuf choices with deployed wire compatibility | Do not hide signedness differences behind a shared alias. Normalize only in a versioned migration with database and protobuf evidence. |
| GL journal/event IDs | `string` | Service-generated opaque IDs, not database sequence numbers | Keep opaque strings. |
| GL source transaction/actor/owner IDs | `string` | Cross-service references supporting multiple owner/source namespaces | Keep strings and explicit source/owner type; converting to numeric IDs would remove namespace flexibility. |
| Generic `base.v1.IdReq`/`IdRes` | `int64` | Existing transport convenience only | Retain for compatibility, but do not use it as a domain-wide ID abstraction. |
A source-level `UserID`, `AssetID`, or generic `ID` package is not approved because:
- repositories are independently versioned and should not acquire a shared-code release dependency;
- protobuf field types are already deployed and inconsistent across legacy contracts;
- aliases would not enforce positivity, ownership, namespace, or existence;
- strong local types are useful only when named for a bounded context and validated there;
- API is an adapter and should map contracts, not become the owner of domain identity types.
## Approved boundary rules
1. Financial values crossing new internal boundaries use canonical base-10 strings
when exact decimal fidelity is required.
2. Each receiving service parses into its own domain amount and applies its own
precision, sign, and business rules.
3. IDs remain explicitly named (`user_id`, `asset_id`, `journal_id`, etc.); generic
IDs must not cross a boundary without the message or operation supplying meaning.
4. Opaque IDs remain strings. Database sequence IDs retain their deployed signedness
until a separately versioned contract migration is justified.
5. `float64` money is allowed only at existing compatibility edges; new domain code
must use exact local values.
6. No service imports another service's domain package. Generated protobuf contracts
and explicit adapter conversions remain the sharing mechanism.
## F002 outcome
There are no approved shared types to introduce. Therefore F002 completes with no
runtime or contract change. The existing Wallet-to-GL canonical decimal string is
already the correct shared representation at the boundary, while both sides retain
their distinct domain types.