CarneiroTech/Content/Cases/en/cnpj-fast-process.md

383 lines
12 KiB
Markdown

---
title: "CNPJ Fast - Alphanumeric CNPJ Migration Process"
slug: "cnpj-fast-process"
summary: "Creation of structured methodology for migrating applications to the new Brazilian alphanumeric CNPJ format, sold to insurance company and collection agency."
client: "Consulting Firm (Internal)"
industry: "Consulting & Digital Transformation"
timeline: "3 months (process creation)"
role: "Solution Architect & Process Designer"
image: ""
tags:
- Process Design
- CNPJ
- Migration Strategy
- Regulatory Compliance
- Consulting
- Sales Enablement
featured: true
order: 3
date: 2024-09-01
seo_title: "CNPJ Fast - Alphanumeric CNPJ Migration Methodology"
seo_description: "Case study of creating a structured process for migrating to Brazilian alphanumeric CNPJ, sold to insurance company and collection agency."
seo_keywords: "CNPJ alphanumeric, migration process, regulatory compliance, consulting, methodology"
---
## Overview
With the introduction of **alphanumeric CNPJ** by the Brazilian Federal Revenue Service, companies faced the challenge of adapting their legacy applications that stored CNPJ as numeric fields (`bigint`, `numeric`, `int`).
I created **CNPJ Fast**, a structured methodology to assess, plan, and execute CNPJ migrations in corporate applications and databases.
**Result:** Process sold to **2 clients** (insurance company and collection agency) before implementation.
---
## Challenge
### Complex Regulatory Change
**Regulatory context:**
- Brazilian Federal Revenue Service introduced **alphanumeric CNPJ**
- CNPJ is no longer just numbers (14 digits)
- Now accepts **letters and numbers** (alphanumeric format)
**Impact on companies:**
```sql
-- BEFORE: Numeric CNPJ
CNPJ BIGINT -- 12345678000190
-- AFTER: Alphanumeric CNPJ
CNPJ VARCHAR(18) -- 12.ABC.678/0001-90
```
**Identified problems:**
1. **Database:** `BIGINT`, `NUMERIC`, `INT` columns don't support characters
2. **Primary keys:** CNPJ used as PK in several tables
3. **Foreign keys:** Relationships between tables
4. **Volume:** Millions of records to migrate
5. **Applications:** Validations, masks, business rules
6. **Testing:** Ensure integrity after migration
7. **Downtime:** Limited maintenance windows
**Without a structured process**, companies risked:
- Data loss
- Database inconsistencies
- Broken applications
- Extended downtime
---
## Solution: CNPJ Fast Process
### 5-Phase Methodology
I designed a structured process that could be replicated across different clients:
```
┌─────────────────────────────────────────────┐
│ PHASE 1: DISCOVERY & ASSESSMENT │
│ - Application inventory │
│ - Database schema analysis │
│ - Identification of impacted tables │
│ - Data volume estimation │
└─────────────────────────────────────────────┘
┌─────────────────────────────────────────────┐
│ PHASE 2: IMPACT ANALYSIS │
│ - Dependency mapping │
│ - Analysis of primary/foreign keys │
│ - Identification of business rules │
│ - Risk assessment │
└─────────────────────────────────────────────┘
┌─────────────────────────────────────────────┐
│ PHASE 3: MIGRATION PLANNING │
│ - Migration strategy (phased commits) │
│ - Automated SQL scripts │
│ - Rollback plan │
│ - Maintenance windows │
└─────────────────────────────────────────────┘
┌─────────────────────────────────────────────┐
│ PHASE 4: EXECUTION │
│ - Batch data migration │
│ - Application updates │
│ - Integration testing │
│ - Integrity validation │
└─────────────────────────────────────────────┘
┌─────────────────────────────────────────────┐
│ PHASE 5: VALIDATION & GO-LIVE │
│ - Regression testing │
│ - Performance validation │
│ - Coordinated go-live │
│ - Post-migration monitoring │
└─────────────────────────────────────────────┘
```
---
### Phase 1: Discovery & Assessment
**Objective:** Understand the complete migration scope
**Deliverables:**
1. **Application Inventory**
- List of applications using CNPJ
- Technologies (ASP 3.0, VB6, .NET, microservices)
- Criticality of each application
2. **Schema Analysis**
```sql
-- Automated discovery script
SELECT
t.TABLE_SCHEMA,
t.TABLE_NAME,
c.COLUMN_NAME,
c.DATA_TYPE,
c.CHARACTER_MAXIMUM_LENGTH
FROM INFORMATION_SCHEMA.TABLES t
JOIN INFORMATION_SCHEMA.COLUMNS c
ON t.TABLE_NAME = c.TABLE_NAME
WHERE c.COLUMN_NAME LIKE '%CNPJ%'
AND c.DATA_TYPE IN ('bigint', 'numeric', 'int')
ORDER BY t.TABLE_SCHEMA, t.TABLE_NAME;
```
3. **Volume Estimation**
- Total records per table
- Size in GB
- Estimated migration time
**Example output:**
| Table | Column | Current Type | Records | Criticality |
|--------|--------|------------|-----------|-------------|
| Clients | CNPJ_Client | BIGINT | 8,000,000 | High |
| Suppliers | CNPJ_Supplier | NUMERIC(14) | 2,500,000 | Medium |
| Transactions | CNPJ_Payer | BIGINT | 90,000,000 | Critical |
---
### Phase 2: Impact Analysis
**Objective:** Map all dependencies and risks
**Key analysis:**
```sql
-- Identifies PKs and FKs involving CNPJ
SELECT
fk.name AS FK_Name,
tp.name AS Parent_Table,
cp.name AS Parent_Column,
tr.name AS Referenced_Table,
cr.name AS Referenced_Column
FROM sys.foreign_keys fk
INNER JOIN sys.tables tp ON fk.parent_object_id = tp.object_id
INNER JOIN sys.foreign_key_columns fkc ON fk.object_id = fkc.constraint_object_id
INNER JOIN sys.columns cp ON fkc.parent_column_id = cp.column_id
AND fkc.parent_object_id = cp.object_id
INNER JOIN sys.tables tr ON fk.referenced_object_id = tr.object_id
INNER JOIN sys.columns cr ON fkc.referenced_column_id = cr.column_id
AND fkc.referenced_object_id = cr.object_id
WHERE cp.name LIKE '%CNPJ%' OR cr.name LIKE '%CNPJ%';
```
**Risk Assessment:**
- **High:** Tables with CNPJ as PK and >10M records
- **Medium:** Tables with FK to CNPJ
- **Low:** Tables without constraints
---
### Phase 3: Migration Planning
**Gradual migration strategy:**
To avoid database locks, I designed a **phased commits** strategy:
```sql
-- Strategy for large tables (>1M records)
-- 1. Add new VARCHAR column
ALTER TABLE Clients
ADD CNPJ_Client_New VARCHAR(18) NULL;
-- 2. Migration in batches (phased commits)
DECLARE @BatchSize INT = 100000;
DECLARE @RowsAffected INT = 1;
WHILE @RowsAffected > 0
BEGIN
UPDATE TOP (@BatchSize) Clients
SET CNPJ_Client_New = FORMAT(CNPJ_Client, '00000000000000')
WHERE CNPJ_Client_New IS NULL;
SET @RowsAffected = @@ROWCOUNT;
WAITFOR DELAY '00:00:01'; -- Pause between batches
END;
-- 3. Remove constraints (PKs, FKs)
ALTER TABLE Clients DROP CONSTRAINT PK_Clients;
-- 4. Rename columns
EXEC sp_rename 'Clients.CNPJ_Client', 'CNPJ_Client_Old', 'COLUMN';
EXEC sp_rename 'Clients.CNPJ_Client_New', 'CNPJ_Client', 'COLUMN';
-- 5. Recreate constraints
ALTER TABLE Clients
ADD CONSTRAINT PK_Clients PRIMARY KEY (CNPJ_Client);
-- 6. Remove old column (after validation)
ALTER TABLE Clients DROP COLUMN CNPJ_Client_Old;
```
**Why this approach?**
- Avoids locking entire table
- Allows pausing/resuming migration
- Minimizes production impact
- Facilitates rollback if needed
---
### Phases 4 & 5: Execution and Validation
**Execution checklist:**
- [ ] Complete database backup
- [ ] Execute migration scripts in batches
- [ ] Update applications (validations, masks)
- [ ] Integration testing
- [ ] Referential integrity validation
- [ ] Performance testing
- [ ] Coordinated go-live
- [ ] 24h post-migration monitoring
---
## Sales Enablement: UX Presentation
**Collaboration with UX Manager:**
The company's UX manager created an **impactful visual presentation** of the CNPJ Fast process:
**Presentation content:**
- Infographics of the 5-phase process
- Examples of time/cost estimates
- Use cases (insurance, banks, fintechs)
- Executive checklist
- Documentation templates
**Result:** Presentation used by sales team for prospecting.
---
## Results & Impact
### Sales Achieved
**Client 1: Insurance Company**
- Stack: ASP 3.0, VB6 components, .NET, microservices
- Scope: Complete legacy application migration
- Status: **Project sold** (execution by another team)
- Value: [Confidential]
**Client 2: Collection Agency**
- Scope: Database migration (~100M records)
- Status: **Project sold and in execution** (by me)
- Particularity: Process was **restructured** to meet specific needs
- See complete case study: [CNPJ Migration - 100M Records](/cases/cnpj-migration-database)
---
### Business Impact
**2 projects sold** before first execution
**Replicable process** for new clients
**Positioning** as specialist in regulatory migrations
**Knowledge base** for future similar projects
---
### Technical Impact
**Tested methodology** in real scenarios
**Reusable documentation** (scripts, checklists, templates)
**Acceleration** of similar projects (from weeks to days)
---
## Tech Stack
`SQL Server` `Migration Strategy` `Process Design` `Regulatory Compliance` `ASP 3.0` `VB6` `.NET` `Microservices` `Batch Processing` `Database Optimization`
---
## Key Decisions & Trade-offs
### Why structured process?
**Alternatives:**
1. Ad-hoc approach per project
2. Manual consulting without methodology
3. **Replicable and scalable process**
**Justification:**
- Reduces Discovery time
- Standardizes deliveries
- Facilitates sales (ready presentation)
- Allows execution by different teams
### Why separate into 5 phases?
**Benefits:**
- Client can approve phase by phase
- Allows adjustments during process
- Facilitates risk management
- Incremental deliveries
---
## Lessons Learned
### 1. UX/Presentation Matters for Sales
The visual presentation made by the UX manager was **crucial** to closing both contracts. Good technical process + poor presentation = no sales.
### 2. Process Sells, Not Just Execution
Creating a **documented methodology** has more commercial value than just offering "consulting hours."
### 3. Each Client is Unique
The client requested **process restructuring**. A good process should be:
- Structured enough to be replicable
- Flexible enough to customize
### 4. Multidisciplinary Collaboration
Working with UX manager (presentations) + sales team (sales) + technical (execution) = success.
---
## Next Steps
**Future opportunities:**
1. **Expansion:** Offer CNPJ Fast to more sectors (banks, fintechs, retail)
2. **Product:** Transform into automated tool (SaaS)
3. **Training:** Enable clients' internal teams
4. **Evolution:** Adapt process for other regulatory migrations (PIX, Open Banking)
---
**Result:** Structured methodology that became a sellable product, generating revenue before the first technical execution.
[Want to implement CNPJ Fast in your company? Get in touch](#contact)