Plano de ImplementaΓ§Γ£o: MetaSet no ArcadiaSuite (Atualizado - 07/04/2026) π VisΓ£o Geral da Arquitetura Atual βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β ARCADIA KERNEL (5001) β β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββββββ β β β Dashboard β β Process β β Health β β LogAggregator β β β β (Web UI) β β Manager β β Monitor β β β β β ββββββββ¬βββββββ ββββββββ¬βββββββ ββββββββ¬βββββββ βββββββββββββββββββ β β β β β β β ββββββββββββββββββ΄βββββββββββββββββ β β β β β βββββββββββββ΄ββββββββββββ β β β Service Registry β β β β (services.json) β β β βββββββββββββ¬ββββββββββββ β β β β β βββββββββββββββββββββββΌββββββββββββββββββββββ β β β β β β β ββββ΄βββ ββββ΄βββ ββββ΄βββ β β β9001 β β9002 β β9003 β + NOVO: 8004 β β βComm β βCore β βWorkerβ + NOVO: 8100 β β βWhatsβ β API β βQueue β (MetaSet) β β βββββββ βββββββ βββββββ β β β β π SERVIΓOS BI: β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β β 8004 β β 8100 β β 8200 β β β β BI-API ββββ MetaSet β β FDB-Bridge β β β β (Node.js) β β (Python) β β (Python) β β β β Gateway β β Superset β β Firebird β β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β ββββββββββββ΄βββββββββββ β PostgreSQL 16 β β βββββββββββββββββ β β β arcadia_db β β β β metaset_db β β β Schema criado e migrado β βββββββββββββββββ β βββββββββββββββββββββββ π§ ConfiguraΓ§Γ£o dos ServiΓ§os (services.json) { "services": [ { "id": "communication", "name": "Communication Service", "type": "node", "command": "node", "args": ["server/communication/index.js"], "cwd": ".", "port": 9001, "autoStart": true, "healthCheck": { "path": "/health", "interval": 30000, "timeout": 5000, "retries": 3 }, "maxRestarts": 5 }, { "id": "core-api", "name": "Core API", "type": "node", "command": "node", "args": ["server/core/index.js"], "cwd": ".", "port": 9002, "autoStart": true, "healthCheck": { "path": "/health", "interval": 30000, "timeout": 5000, "retries": 3 }, "maxRestarts": 5 }, { "id": "bi-api", "name": "BI API Gateway", "type": "node", "command": "node", "args": ["server/bi/index.js"], "cwd": ".", "port": 8004, "autoStart": true, "env": { "METASET_PORT": "8100", "METASET_HOST": "127.0.0.1", "FDB_BRIDGE_PORT": "8200" }, "healthCheck": { "path": "/bi/health", "interval": 30000, "timeout": 5000, "retries": 3 }, "maxRestarts": 5, "description": "Gateway de BI com proxy para MetaSet e FDB-Bridge" }, { "id": "metaset", "name": "MetaSet BI", "type": "python", "command": "python", "args": ["server/bi/metaset/run.py", "--port=8100", "--host=127.0.0.1"], "cwd": ".", "port": 8100, "autoStart": false, "env": { "PYTHONPATH": "server/bi/metaset", "FLASK_ENV": "production", "SUPERSET_CONFIG_PATH": "server/bi/metaset/superset_config.py", "SUPERSET_SECRET_KEY": "${SUPERSET_SECRET_KEY}" }, "healthCheck": { "path": "/health", "interval": 30000, "timeout": 10000, "retries": 3 }, "maxRestarts": 3, "description": "Apache Superset rebrandado como MetaSet (Docker-managed)" }, { "id": "fdb-bridge", "name": "FDB Bridge", "type": "python", "command": "python", "args": ["server/bi/fdb-bridge/main.py", "--port=8200"], "cwd": ".", "port": 8200, "autoStart": false, "env": { "PYTHONPATH": "server/bi/fdb-bridge", "FDB_HOST": "${FDB_HOST}", "FDB_PORT": "${FDB_PORT:-3050}" }, "healthCheck": { "path": "/health", "interval": 60000, "timeout": 5000, "retries": 3 }, "maxRestarts": 3, "description": "Conector Firebird para sincronizaΓ§Γ£o de dados (Docker-managed)" } ] } ποΈ Estrutura de DiretΓ³rios Final arcadiasuite/ βββ server/ β βββ kernel/ # β Kernel finalizado β β βββ core/ β β β βββ ProcessManager.ts # β Suporta 'python' e 'node' β β β βββ HealthMonitor.ts β β β βββ LogAggregator.ts β β βββ dashboard/ # Dashboard Web UI (porta 5001) β β βββ config/ β β β βββ services.json # β ConfiguraΓ§Γ£o dos serviΓ§os β β βββ index.ts β β β βββ bi/ # π MΓDULO BI COMPLETO β β βββ index.ts # BI-API Gateway (porta 8004) β β βββ routes/ β β β βββ query.ts # Queries nativas Drizzle β β β βββ export.ts # ExportaΓ§Γ£o PDF/Word β β β βββ sync.ts # SincronizaΓ§Γ£o FDB β β β β β βββ metaset/ # π METASET (Superset) β β β βββ run.py # Entry point Python β β β βββ superset_config.py # ConfiguraΓ§Γ£o Arcadia β β β βββ security_manager.py # ArcadiaSecurityManager β β β βββ requirements.txt # DependΓͺncias Python β β β βββ Dockerfile # Multi-stage build β β β βββ init.sh # Container init script β β β βββ branding/ # π¨ Assets MetaSet β β β β βββ metaset-logo-horiz.png β β β β βββ metaset-icon.png β β β β βββ favicon-16x16.png β β β β βββ favicon-32x32.png β β β β β β β βββ superset-src/ # Fork Apache Superset 4.1.0 β β β βββ superset-frontend/ # β Rebuild com branding MetaSet β β β β βββ src/ β β β β βββ features/home/RightMenu.tsx # β Textos atualizados β β β β βββ components/ErrorMessage/OAuth2RedirectMessage.tsx β β β β βββ pages/... # β TraduΓ§Γ΅es atualizadas β β β β β β β βββ superset/ β β β β βββ config.py # β APP_NAME = "MetaSet" β β β β βββ constants.py # β USER_AGENT = "MetaSet" β β β β βββ cli/ β β β β βββ templates/ # β HTML templates atualizados β β β β βββ translations/ # β TraduΓ§Γ΅es en/pt_BR β β β βββ ... β β β β β βββ fdb-bridge/ # π FIREBIRD CONNECTOR (skeleton) β β βββ main.py # ServiΓ§o Python porta 8200 β β βββ sync.py # LΓ³gica de sincronizaΓ§Γ£o β β βββ models.py # Modelos SQLAlchemy β β βββ requirements.txt β β β βββ ... (outros serviΓ§os) β βββ client/ β βββ src/ β βββ pages/ β βββ bi/ # π FRONTEND BI β βββ index.tsx # Dashboard BI Arcadia β βββ metaset/ # Embed MetaSet β β βββ Dashboard.tsx β β βββ SQLLab.tsx β β βββ Explore.tsx β βββ components/ β βββ QueryBuilder.tsx β βββ ExportButton.tsx β βββ FDBSyncStatus.tsx β βββ branding/ # π ASSETS COMPARTILHADOS βββ logos/ β βββ arcadia-suite-logo.svg β βββ arcadia-suite-logo-horiz.svg β βββ metaset-logo.svg β βββ metaset-icon.svg βββ colors/ β βββ arcadia-palette.json βββ fonts/ βββ Inter-Variable.ttf π Arquitetura de SeguranΓ§a & Multi-tenancy Fluxo de AutenticaΓ§Γ£o βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β Cliente ββββββΆβ Arcadia ββββββΆβ BI-API ββββββΆβ MetaSet β β (React) β β Login β β (8004) β β (8100) β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β β β Headers: β SET LOCAL β X-Tenant-ID: 123 β app.current_tenant β X-User-Role: admin β = 123; β Authorization: JWT β βΌ βΌ βββββββββββββββ βββββββββββββββ β RLS Check β β PostgreSQL β β Middleware β β (RLS) β βββββββββββββββ βββββββββββββββ ArcadiaSecurityManager (server/bi/metaset/security_manager.py) β CORRIGIDO """ Security Manager integrado ao Arcadia Kernel - Extrai tenant e usuΓ‘rio do token JWT - Aplica RLS automaticamente - Sincroniza roles com ArcadiaSuite - Preserva comportamento padrΓ£o do Superset (chama super().before_request()) """ from superset.security import SupersetSecurityManager from flask import g, request, current_app from sqlalchemy import text import jwt import logging logger = logging.getLogger(__name__) class ArcadiaSecurityManager(SupersetSecurityManager): ROLE_MAPPING = { 'superadmin': 'Admin', 'admin': 'Alpha', 'analyst': 'Gamma', 'viewer': 'Public', } def before_request(self): """Executado antes de cada requisiΓ§Γ£o - preserva comportamento padrΓ£o""" super().before_request() # β CRΓTICO: preserva autenticaΓ§Γ£o padrΓ£o auth_header = request.headers.get('Authorization', '') if not auth_header.startswith('Bearer '): return token = auth_header.replace('Bearer ', '') try: payload = jwt.decode( token, current_app.config.get('SECRET_KEY', 'default-key'), algorithms=['HS256'] ) g.tenant_id = payload.get('tenant_id') g.user_id = payload.get('sub') g.user_role = payload.get('role', 'viewer') g.user_email = payload.get('email') if g.tenant_id: try: from superset import db db.session.execute( text(f"SET LOCAL app.current_tenant = {g.tenant_id}") ) except Exception as e: logger.warning(f"Erro ao aplicar RLS: {e}") except jwt.ExpiredSignatureError: logger.warning("JWT token expirado") except jwt.InvalidTokenError: logger.warning("JWT token invΓ‘lido") except Exception as e: logger.warning(f"Erro ao processar JWT: {e}") π CΓ³digo dos ServiΓ§os Python 1. MetaSet Entry Point (server/bi/metaset/run.py) """ MetaSet - ServiΓ§o Apache Superset integrado ao Arcadia Kernel Porta: 8100 (Docker container) Gunicorn: 2 workers, gthread, 120s timeout """ import os import sys import logging from pathlib import Path logging.basicConfig( level=logging.INFO, format='[MetaSet] %(levelname)s: %(message)s', handlers=[logging.StreamHandler(sys.stdout)] ) logger = logging.getLogger(__name__) def create_app(): """Criar aplicaΓ§Γ£o Flask do Superset com endpoints customizados""" from superset import create_app as create_superset_app from datetime import datetime app = create_superset_app() @app.route('/health') def health_check(): return {'status': 'healthy', 'service': 'metaset'}, 200 @app.route('/info') def service_info(): return { 'name': 'MetaSet', 'description': 'Business Intelligence by ArcadiaSuite', 'version': app.config.get('VERSION_STRING', '4.1.0'), 'features': ['sql_lab', 'dashboards', 'charts', 'rls', 'embedding'] } return app # MetaSetApplication: custom Gunicorn app # Config: 2 workers, gthread, 120s timeout, stdout/stderr logging 2. BI-API Gateway (server/bi/index.ts) """ * BI-API Gateway * Porta: 8004 * Integra MetaSet (8100) e FDB-Bridge (8200) ao ArcadiaSuite * Proxy manual usando mΓ³dulo http nativo (bypass http-proxy-middleware ECONNRESET) """ import express from 'express'; import http from 'http'; const METASET_URL = `http://${process.env.METASET_HOST || 'metaset'}:${process.env.METASET_PORT || '8100'}`; // Manual proxy: http.request() com pathRewrite // SoluΓ§Γ£o para ECONNRESET com Gunicorn keepalive // Health check agrega MetaSet + FDB-Bridge status π¨ ConfiguraΓ§Γ£o de Branding (superset_config.py) """ ConfiguraΓ§Γ£o do MetaSet (Superset) para ArcadiaSuite """ import os from pathlib import Path BASE_DIR = Path(__file__).parent.absolute() SUPERSET_HOME = BASE_DIR / 'superset_home' # ========================================== # BRANDING - ARCADIASUITE β IMPLEMENTADO # ========================================== APP_NAME = "MetaSet by ArcΓ‘dia" APP_ICON = "/static/assets/images/branding/metaset-logo-horiz.png" LOGO_TARGET_PATH = "/" LOGO_TOOLTIP = "MetaSet - Business Intelligence by ArcadiaSuite" # Favicons FAVICONS = [ {"href": "/static/assets/images/branding/favicon-16x16.png", "sizes": "16x16"}, {"href": "/static/assets/images/branding/favicon-32x32.png", "sizes": "32x32"}, {"href": "/static/assets/images/branding/apple-touch-icon.png", "sizes": "180x180"}, ] # Cores ArcadiaSuite ARCADIA_PRIMARY = "#0ea5e9" # Azul celeste ARCADIA_SECONDARY = "#10b981" # Verde ARCADIA_ACCENT = "#f59e0b" # Laranja ARCADIA_BG = "#f8fafc" # Background claro ARCADIA_TEXT = "#1e293b" # Texto escuro # Desabilitar cache de arquivos estΓ‘ticos SEND_FILE_MAX_AGE_DEFAULT = 0 THEME_DEFAULT = { "token": { "brandAppName": "MetaSet", "brandLogoAlt": "MetaSet by ArcadiaSuite", "brandLogoUrl": "/static/assets/images/branding/metaset-logo-horiz.png", "brandLogoMargin": "16px 0", "brandLogoHeight": "28px", "colorPrimary": ARCADIA_PRIMARY, "colorLink": ARCADIA_PRIMARY, "colorSuccess": ARCADIA_SECONDARY, "colorWarning": ARCADIA_ACCENT, "colorError": "#ef4444", "colorInfo": "#3b82f6", "colorBgLayout": ARCADIA_BG, "colorBgContainer": "#ffffff", "colorText": ARCADIA_TEXT, "colorTextSecondary": "#64748b", "fontFamily": "Inter, -apple-system, BlinkMacSystemFont, sans-serif", "fontSize": 14, "borderRadius": 8, "borderRadiusLG": 12, "boxShadow": "0 1px 3px 0 rgba(0, 0, 0, 0.1)", "boxShadowSecondary": "0 4px 6px -1px rgba(0, 0, 0, 0.1)", }, "algorithm": "default", } THEME_DARK = { ... } # Tema escuro # ========================================== # SEGURANΓA & MULTI-TENANCY # ========================================== from security_manager import ArcadiaSecurityManager CUSTOM_SECURITY_MANAGER = ArcadiaSecurityManager # β Funcionando AUTH_TYPE = 1 WTF_CSRF_ENABLED = True WTF_CSRF_TIME_LIMIT = 3600 SESSION_COOKIE_HTTPONLY = True SESSION_COOKIE_SECURE = False # True em produΓ§Γ£o com HTTPS SESSION_COOKIE_SAMESITE = "Lax" # ========================================== # BANCO DE DADOS # ========================================== SQLALCHEMY_DATABASE_URI = os.getenv( 'DATABASE_URL', 'postgresql://arcadia:arcadia123@db:5432/metaset_db' ) SQLALCHEMY_ENGINE_OPTIONS = { "connect_args": {"options": "-c row_security=on"}, "pool_pre_ping": True, "pool_recycle": 300, } # ========================================== # FEATURE FLAGS # ========================================== FEATURE_FLAGS = { "EMBEDDED_SUPERSET": True, "EMBEDDABLE_CHARTS": True, "DASHBOARD_RBAC": True, "SQLLAB_BACKEND_PERSISTENCE": True, "DASHBOARD_VIRTUALIZATION": True, "DRILL_BY": True, "CSS_TEMPLATES": True, "ENABLE_JAVASCRIPT_CONTROLS": False, "ALERT_REPORTS": False, } π Roadmap de ImplementaΓ§Γ£o (8 Semanas) Semana Foco EntregΓ‘veis 1 Setup MetaSet Fork Superset, branding bΓ‘sico (logo, cores), superset_config.py 2 IntegraΓ§Γ£o Kernel services.json atualizado, BI-API gateway, proxy funcionando 3 SeguranΓ§a ArcadiaSecurityManager, JWT validation, injeΓ§Γ£o de headers 4 Multi-tenancy RLS PostgreSQL, isolamento de datasources, testes de seguranΓ§a 5 FDB Bridge ServiΓ§o Python porta 8200, conexΓ£o Firebird, testes de sync 6 Sync Engine SincronizaΓ§Γ£o incremental, agendamento, resoluΓ§Γ£o de conflitos 7 Frontend PΓ‘ginas BI no React, embed de dashboards, SQL Lab integrado 8 Go-Live Testes E2E, documentaΓ§Γ£o, otimizaΓ§Γ£o de performance β Checklist de ImplementaΓ§Γ£o - STATUS ATUAL Semana 1-2: Setup e IntegraΓ§Γ£o β CONCLUΓDO β Clonar superset (4.1.0rc4) para server/bi/metaset/superset-src β Criar superset_config.py com branding ArcadiaSuite (APP_NAME, cores, tema) β Rebuild do frontend React com logos MetaSet (COMPLETO - 07/04/2026) - SubstituiΓ§Γ£o de textos: "Superset" β "MetaSet" em todo codebase - TraduΓ§Γ΅es atualizadas: en/LC_MESSAGES/messages.po, pt_BR/LC_MESSAGES/messages.po - Arquivos fonte atualizados: RightMenu.tsx, OAuth2RedirectMessage.tsx, SliceHeaderControls, HeaderActionsDropdown, AnchorLink, EmbeddedModal, useExploreAdditionalActionsMenu - Templates HTML: basic.html, public_welcome.html, 404.html, 500.html - Config Python: config.py (APP_NAME), constants.py (USER_AGENT), cli/*.py - Build webpack concluΓdo com sucesso (429 warnings, 0 errors) - Assets compilados com branding MetaSet incorporado β Criar server/bi/metaset/run.py entry point (Gunicorn) β Criar server/bi/metaset/init.sh container initialization β Criar server/bi/index.ts BI-API gateway (proxy manual Node.js) β Atualizar server/kernel/config/services.json (autoStart: false, Docker Discovery) β Docker Compose: metaset + bi-api + redes configuradas β Dockerfile multi-stage build com assets locais (nΓ£o mais copiando de apache/superset:latest) β Deploy: https://bi.onboardbi.com.br (MetaSet rodando com branding completo) β Container antigo arcadia-prod-superset-1 removido β Container metaset conectado Γ rede coolify para Traefik β Banco de dados metaset_db migrado e inicializado β UsuΓ‘rio admin criado (login: admin / senha: admin) β BI-API Gateway proxy /bi/metaset funcionando (porta 8004) β Health checks integrados (metaset + bi-api) β requirements.txt corrigido (compatΓvel apache-superset 4.1.0) β extra_hosts configurado para resolver db β 10.0.10.2 (IPv4) Semana 3-4: SeguranΓ§a π EM ANDAMENTO β Implementar ArcadiaSecurityManager (CORRIGIDO - chama super().before_request()) β Configurar JWT validation π Criar schema metaset no PostgreSQL (banco criado, migraΓ§Γ΅es aplicadas) β Configurar RLS em tabelas do arcadia_db β Testar isolamento entre tenants Semana 5-6: FDB Bridge β³ PENDENTE β Criar server/bi/fdb-bridge/main.py β Implementar conexΓ£o Firebird (fdb library) β Criar lΓ³gica de sincronizaΓ§Γ£o incremental β Configurar agendador APScheduler β Testar sync Matriz + 5 Filiais Semana 7-8: Frontend e Go-Live β³ PENDENTE β Criar pΓ‘ginas BI no frontend React β Implementar embed de dashboards MetaSet β Criar componente de status de sincronizaΓ§Γ£o β Testes E2E completos β DocumentaΓ§Γ£o do mΓ³dulo BI π LiΓ§Γ΅es Aprendidas (07/04/2026) 1. Frontend rebuild obrigatΓ³rio: Substituir arquivos estΓ‘ticos PNG no Docker nΓ£o funciona porque os assets sΓ£o compilados em bundles JS pelo webpack. Γ necessΓ‘rio rebuildar o superset-frontend com npm run build. 2. DependΓͺncias faltantes: O build pode falhar por pacotes nΓ£o instalados: @react-spring/web, global-box, currencyformatter.js. Instalar com npm install --legacy-peer-deps. 3. Security Manager: O ArcadiaSecurityManager DEVE chamar super().before_request() no inΓcio do mΓ©todo para preservar o comportamento de autenticaΓ§Γ£o padrΓ£o do Flask-AppBuilder. Sem isso, o login falha com AttributeError: user. 4. IPv6 vs IPv4: O hostname "db" resolve para IPv6 (fd20:a15a:c2d4::c) mas o psycopg2 falha na autenticaΓ§Γ£o via IPv6. SoluΓ§Γ£o: mapear db β IP IPv4 via extra_hosts no docker-compose.yml. 5. Container antigo: Sempre verificar se hΓ‘ containers antigos com labels do Traefik apontando para o mesmo domΓnio. Eles podem conflitar com o novo container. 6. Traefik discovery: O container DEVE estar na mesma rede Docker que o Traefik (coolify) para ser descoberto automaticamente. π° Custos Adicionais Componente EspecificaΓ§Γ£o Custo Estimado MemΓ³ria extra +2GB para Python (MetaSet + FDB-Bridge) ~$20/mΓͺs Storage PostgreSQL +10GB para metadata MetaSet ~$5/mΓͺs CPU Processamento de sync FDB ~$15/mΓͺs Total ~$40/mΓͺs π Comandos de Deploy # 1. Build da imagem Docker cd /opt/arcadia_merged docker compose build --no-cache metaset # 2. Recriar container com nova imagem docker compose up -d --force-recreate metaset # 3. Verificar status curl http://localhost:8100/health curl https://bi.onboardbi.com.br/login/ | grep "