VideoStudy/REFERENCES.md
2026-02-09 18:31:08 -03:00

22 KiB

VideoStudy.app - Phase 1: References Analysis

Date: 2026-02-06 Objective: Document architecture, dependencies, and design patterns from reference projects


1. YTExtractor Analysis

1.1 Project Overview

  • Type: ASP.NET Core Web API
  • Framework: .NET 8.0
  • Purpose: Extract YouTube video metadata and transcripts
  • Deployment: Docker container with Linux support

1.2 Services Architecture

A. YoutubeService (Primary)

  • Purpose: YouTube video extraction using yt-dlp command-line tool
  • Key Methods:
    • IsValidYouTubeUrl(string url) - Regex validation
    • GetVideoInfo(string url, string workingDir) - Extract metadata (title, thumbnail)
    • GetSubtitles(string url, string language, string workingDir) - Download captions in VTT format
  • Execution Model: System.Diagnostics process execution (yt-dlp)
  • Platform Support: Windows (.exe) and Linux/macOS (command)

B. YoutubeDataService (Alternative)

  • Purpose: Google YouTube API v3 integration
  • Methods: Same interface as YoutubeService
  • API Key: Environment variable YOUTUBE_API_KEY
  • Status: Currently not integrated in main endpoint (backup option)

C. ConvertTranscriptService

  • Purpose: VTT transcript conversion and cleaning
  • Key Methods:
    • ExtractPlainText(string vttContent) - Remove timestamps, tags, music markers
    • ConvertToSrt(string vttContent) - VTT → SRT format conversion
  • Processing: 7-step regex-based cleaning pipeline
    • Remove WEBVTT header
    • Remove timestamps (HH:MM:SS.mmm format)
    • Strip HTML/VTT style tags (<00:00:00.280>, <c>)
    • Remove music brackets [Música]
    • Remove positioning markers
    • Remove all remaining tags
    • Normalize whitespace

D. MongoDBConnector

  • Purpose: Data persistence layer
  • Database: MongoDB "YTExtractor" database, "videos" collection
  • Methods:
    • CRUD operations (Insert, Get, Update, Delete)
    • GetVideoByUrl(string url) - Caching mechanism
  • Caching Strategy: Check cache before download to prevent redundant operations

1.3 API Endpoint

POST /api/video-info

Request:
{
  "url": "https://www.youtube.com/watch?v=VIDEO_ID",
  "language": "en"
}

Response:
{
  "url": "...",
  "title": "...",
  "thumbnailUrl": "...",
  "subtitles": "Plain text transcript..."
}

1.4 Data Flow

HTTP POST → Validate URL → Check MongoDB cache
  ├─ FOUND: Return cached response
  └─ NOT FOUND:
      ├─ Execute yt-dlp (metadata) → Get title + thumbnail
      ├─ Execute yt-dlp (subtitles) → Download .vtt file
      ├─ Clean VTT → Plain text via ConvertTranscriptService
      ├─ Store in MongoDB
      └─ Return response

1.5 Key Dependencies

Package Version Purpose
Google.Apis.YouTube.v3 1.69.0.3707 YouTube API client (backup)
MongoDB.Driver 3.1.0 Database access
MongoDB.Bson 3.1.0 BSON serialization
Serilog.AspNetCore 9.0.0 Structured logging
Serilog.Sinks.Seq 9.0.0 Centralized logging (Seq server)
Swashbuckle.AspNetCore 6.6.2 Swagger/OpenAPI UI

1.6 Configuration

{
  "MongoDbConnection": "mongodb://localhost:27017",
  "Serilog": {
    "MinimumLevel": "Information",
    "Sinks": ["Console", "Seq"],
    "Properties": {
      "Application": "YTExtractor",
      "Workspace": "Dev"
    }
  }
}

1.7 External Dependencies

  • yt-dlp - Command-line video tool (included in Docker image)
  • MongoDB - Document database
  • Seq Server - Log aggregation (dev only, optional)

1.8 Design Patterns Used

  • Repository Pattern - MongoDBConnector
  • Service Layer - Business logic separation
  • Process Execution - System.Diagnostics for yt-dlp
  • Caching - MongoDB-based result caching
  • Regex Parsing - Text processing pipeline

2. vcart.me.novo Analysis

2.1 Project Overview

  • Type: Full-stack ASP.NET Core MVC application
  • Framework: .NET 8.0
  • Purpose: LinkTree clone (business card/link aggregation platform)
  • CSS Framework: Bootstrap 5.3.2 + Custom CSS

2.2 CSS Architecture

A. Framework

  • Primary: Bootstrap 5.3.2 (grid, components, utilities)
  • Approach: Custom CSS + Bootstrap utilities (NO Tailwind, NO SCSS/LESS)
  • Preprocessor: Vanilla CSS only
  • CSS Variables: Custom properties for theming

B. Design Tokens

Color Palette:

/* Primary */
--primary-color: #007bff;           /* Bootstrap blue */
--secondary-color: #0056b3;         /* Darker shade */

/* Accents */
--accent-purple: #764ba2;           /* Gradients */
--accent-blue: #667eea;

/* Semantic */
--success: #28a745;
--warning: #ffc107;
--danger: #dc3545;

/* Text & Backgrounds */
--text-color: #212529;
--bg-light: #f8f9fa;
--bg-white: #ffffff;
--border-color: #dee2e6;

/* Dark Mode */
@media (prefers-color-scheme: dark) {
  --bg-dark: #121212;
  --bg-card-dark: rgba(33, 37, 41, 0.95);
}

Gradients:

/* Hero section gradient */
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);

/* Loading shimmer animation */
background: linear-gradient(90deg, rgba(255,255,255,0.1), rgba(255,255,255,0.3), rgba(255,255,255,0.1));

C. Typography System

Font Sizes (Responsive):

  • Base: 14px (mobile) → 16px (768px+)
  • Display (Hero): 3.5rem
  • Headings: Scales responsively
  • Body/Links: 0.9rem - 1.2rem

Font Weights:

  • Regular: 400
  • Medium: 500 (buttons, links)
  • Semi-bold: 600 (titles)
  • Bold: 700 (headings)

Line Heights:

  • Standard: 1.5
  • Relaxed: 1.6

D. Component Styles

Buttons:

.btn {
  border-radius: 50px;              /* Pill shape */
  font-weight: 500;
  padding: 0.6rem 1.5rem;
  transition: all 0.3s ease;
}

.btn-primary:hover {
  transform: translateY(-2px);
  box-shadow: 0 5px 15px rgba(37, 140, 251, 0.3);
}

Cards:

.card {
  border-radius: 15px;
  box-shadow: 0 8px 32px rgba(0, 0, 0, 0.1);
  overflow: hidden;
}

.profile-card {
  background-color: rgba(255, 255, 255, 0.95);
  backdrop-filter: blur(10px);      /* Glassmorphism */
}

Forms:

.form-control {
  border-radius: 10px;
  border: 2px solid #e9ecef;
  transition: all 0.3s ease;
}

.form-control:focus {
  border-color: #007bff;
  box-shadow: 0 0 0 0.2rem rgba(0, 123, 255, 0.25);
}

Badges & Alerts:

.badge {
  border-radius: 20px;              /* Pill shaped */
}

.alert {
  border-radius: 15px;
  border: none;
}

E. Layout Patterns

Spacing System:

  • Bootstrap utilities (m-, p-, gap)
  • Custom: 1rem, 1.5rem, 2rem
  • Mobile-first approach

Breakpoints:

xs: Default (mobile-first)
sm: 576px+
md: 768px+
lg: 992px+
xl: 1200px+

Flexbox Patterns:

  • Navigation: flex row with gap
  • Profile cards: centered flex column
  • Link buttons: block-level

Hero Section:

.hero-section {
  background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
  color: white;
  padding: 2rem (mobile) - 5rem (desktop);
}

2.3 Frontend Structure

Key Pages:

  • Home/Index - Landing page
  • User Pages - Public profiles
  • Dashboard - Management interface
  • Manage Page - Page editor (complex)
  • Pricing - Subscription tiers
  • Authentication - Login/OAuth
  • Admin - Admin dashboard
  • Moderation - Content review

View Components:

  • ModerationMenu - Conditional navigation
  • SupportFab - Floating Action Button
  • _ThemeStyles - Dynamic CSS generation

JavaScript (Vanilla, no frameworks):

  • site.js - Global utilities (slug generation, validation, toasts)
  • support-fab.js - Support button
  • cookie-consent.js - Cookie banner
  • email-handler.js - Form handling
  • rating.js - Rating modal

2.4 Backend Architecture

Project Structure:

Controllers/     (16 MVC controllers)
Views/          (30 Razor templates)
Models/         (15 domain entities)
Services/       (39 business logic services)
Repositories/   (12 data access layer)
Areas/          (Artigos, Tutoriais, Support)
Middleware/     (Custom request handling)

Key Services:

  • UserPageService, ModerationService, PaymentService
  • ThemeService (dynamic CSS generation)
  • AuthService, EmailService, ImageStorageService
  • LivePageService, DowngradeService, TrialExpirationService
  • PlanConfigurationService

Database: MongoDB 7.0

  • Collections: users, userpages, categories, subscriptions
  • GridFS for file storage
  • Compound indexes for optimization

2.5 Key Dependencies

Package Version Purpose
MongoDB.Driver 3.4.2 Database driver
Stripe.net 48.4.0 Payment processing
Microsoft.AspNetCore.Authentication.Google 8.0.4 OAuth
SendGrid 9.29.3 Email service
SixLabors.ImageSharp.Web 3.1.0 Image manipulation
Serilog 8.0.0 Structured logging
Serilog.Sinks.OpenSearch 1.3.0 Log aggregation
Markdig 0.43.0 Markdown parsing
HtmlAgilityPack 1.11.54 HTML parsing
YamlDotNet 16.3.0 YAML configuration
Bootstrap 5.3.2 CSS framework
jQuery 3.7.1 DOM utilities
Font Awesome 6.4.0 Icons (CDN)

2.6 Design Patterns

  • Glassmorphism - Semi-transparent cards with backdrop blur
  • Micro-interactions - Smooth transitions (0.2s-0.5s)
  • Gradient accents - Purple-blue gradients
  • Responsive typography - Fluid font sizing
  • Color psychology - Blue for trust, purple for premium
  • Accessibility - Dark mode, reduced motion, high contrast

2.7 Build & Deployment

Build Targets:

  • Debug, Release, Testing configurations
  • Linux x64, ARM64 (Docker support)
  • Publishing profiles included

Static Assets:

  • Version-controlled via asp-append-version="true"
  • Content folder for Markdown
  • Resources folder for localization

3. ChatAPI / ChatRAG Analysis

3.1 Project Overview

ChatAPI:

  • Type: ASP.NET Core Web API
  • Framework: .NET 8.0
  • Purpose: Classification-based chat routing (Chat/Company/HR)
  • LLM Provider: OpenChat 3.5 via ServerSpace

ChatRAG:

  • Type: ASP.NET Core Web API
  • Framework: .NET 8.0
  • Purpose: RAG system with vector search and document retrieval
  • LLM Provider: Groq (llama-3.1-8b-instant)

3.2 Semantic Kernel Integration

Version

  • Semantic Kernel: 1.26.0 (current latest)
  • Connectors:
    • Microsoft.SemanticKernel.Connectors.OpenAI 1.26.0 (ChatGPT, Groq, OpenChat)
    • Microsoft.SemanticKernel.Connectors.Ollama 1.26.0-alpha (Local Ollama)
    • Microsoft.SemanticKernel.Connectors.Google 1.26.0-alpha (Gemini)
    • Lost.SemanticKernel.Connectors.Anthropic 1.25.0-alpha3 (Claude)

Registration Pattern

// In Program.cs
builder.Services.AddKernel();

// Add LLM provider (choose one):
// OpenAI-compatible (OpenChat, Groq, DeepInfra):
builder.Services.AddOpenAIChatCompletion(
    modelId: "openchat-3.5-0106",
    endpoint: new Uri("https://gpt.serverspace.com.br/v1/chat/completions"),
    apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY")
);

// Or Ollama (local):
builder.Services.AddOllamaChatCompletion(
    modelId: "llama3.1:latest",
    endpoint: new Uri("http://localhost:11434")
);

// Or Google Gemini:
builder.Services.AddGoogleAIGeminiChatCompletion(
    modelId: "gemini-1.5-flash-latest",
    apiKey: Environment.GetEnvironmentVariable("GOOGLE_API_KEY")
);

// Add text embeddings (recommend Ollama):
builder.Services.AddOllamaTextEmbeddingGeneration(
    modelId: "all-minilm",
    endpoint: new Uri("http://localhost:11434")
);

3.3 Stepwise Planner Status

ANSWER: NO - Stepwise Planner is NOT used

Reason: Stepwise Planner is designed for multi-step function calling workflows. This architecture uses RAG with hierarchical query analysis instead.

Alternative Orchestration Pattern:

ChatAPI: Simple classification routing

  • Query Classification → Strategy Selection → Response Service Selection
  • Example: "RH salary question" → RH Service → ChatBotRHCall → External API

ChatRAG: Hierarchical RAG pipeline

  • Query Analysis (intent detection + strategy selection)
  • Multi-stage search (overview/specific/detailed/out-of-scope)
  • Confidence verification
  • Response generation with context assembly

3.4 Provider Switching Mechanism

Method: Configuration-driven with commented alternatives in Program.cs

ChatAPI Configuration:

// Active: OpenChat 3.5
builder.Services.AddOpenAIChatCompletion(
    "openchat-3.5-0106",
    new Uri("https://gpt.serverspace.com.br/v1/chat/completions"),
    "API_KEY"
);

// Available alternatives (commented):
// Local Ollama: builder.Services.AddOllamaChatCompletion("llama3.1:latest", new Uri("http://192.168.0.150:11434"));
// OpenAI GPT-4: builder.Services.AddOpenAIChatCompletion("gpt-4o-mini", "sk-proj-...");
// Google Gemini: builder.Services.AddGoogleAIGeminiChatCompletion("gemini-1.5-flash-latest", "AIzaSy...");
// Claude: builder.Services.AddAnthropicChatCompletion("claude-3-5-sonnet-latest", "sk-ant-...");

// Embeddings: Ollama
builder.Services.AddOllamaTextEmbeddingGeneration("all-minilm", new Uri("http://192.168.0.150:11434"));

ChatRAG Configuration:

// Active: Groq
var model = "llama-3.1-8b-instant";
var url = "https://api.groq.com/openai/v1";
builder.Services.AddOpenAIChatCompletion(model, new Uri(url), key);

// Embeddings: Ollama
builder.Services.AddOllamaTextEmbeddingGeneration("all-minilm", new Uri("http://localhost:11434"));

Switch Process:

  1. Comment out current provider line
  2. Uncomment desired provider
  3. Update model ID and endpoint
  4. Provide API key (environment variable or secret)
  5. Run: dotnet build && dotnet run

3.5 RAG Implementation (ChatRAG)

Vector Database Support

Pluggable via Factory Pattern:

  1. Qdrant (Primary) - High-performance vector DB
  2. MongoDB (Secondary) - Embedded vectors in documents
  3. Chroma (Alternative) - Lightweight vector DB
  4. Hybrid - Combine multiple sources

Configuration:

"VectorDatabase": {
  "Provider": "Hybrid",
  "Qdrant": {
    "Host": "localhost",
    "Port": 6334,
    "CollectionName": "texts-whats",
    "VectorSize": 384,
    "Distance": "Cosine"
  }
}

RAG Strategies

HierarchicalRAGService: 4-stage search

  1. Overview - Fetch all docs, categorize, summarize
  2. Specific - Direct similarity search + context expansion
  3. Detailed - Multi-concept search + knowledge gap filling
  4. Out-of-scope - Fallback for unrelated queries

ConfidenceAwareRAGService: Multi-dimensional RAG

  • Intent detection (comparative, definition)
  • Heuristic search with query candidates
  • Dynamic confidence threshold adjustment
  • Confidence relaxation for specific intent types

Document Ingestion Pipeline

Document Upload
    ↓
Validation & Sanitization
    ↓
Auto-Enrichment (Groq LLM) - Optional
    ├─ Generate aliases
    ├─ Extract topics
    └─ Create summary
    ↓
Metadata Generation
    ├─ Title normalization
    ├─ Acronym extraction
    ├─ Search term building (no diacritics)
    └─ Structured metadata
    ↓
Vector Embedding
    └─ Ollama all-minilm (384 dimensions)
    ↓
Store in Vector Database
    ├─ Document content
    ├─ Metadata
    ├─ Vector embedding
    └─ Searchable indices

3.6 Services Architecture

ChatAPI Services:

  • ResponseChatService - General chat
  • ResponseCompanyService - Company queries
  • ResponseBotRHService - HR integration
  • ChatHistoryService - Conversation memory
  • ClassifierPersistence - Query classification storage
  • ResponseFactory - Service selection

ChatRAG Services:

  • ConfidenceAwareRAGService - Confidence-verified RAG
  • HierarchicalRAGService - Multi-strategy RAG
  • QdrantVectorSearchService - Qdrant operations
  • MongoVectorSearchService - MongoDB vectors
  • ChromaVectorSearchService - Chroma operations
  • DocumentIngestionService - Document preprocessing
  • PromptConfigurationService - Domain/language templates
  • ConfidenceVerifier - Quality assurance metrics
  • VectorDatabaseFactory - Provider instantiation

3.7 Key Interfaces (Semantic Kernel)

// Core abstractions
IChatCompletionService              // LLM chat operations
ITextEmbeddingGenerationService     // Vector embedding generation
IVectorSearchService                // Vector database queries
ITextDataService                    // Document storage operations

3.8 Key Dependencies

Common:

Package Version Purpose
Microsoft.SemanticKernel 1.26.0 Core Semantic Kernel
Microsoft.SemanticKernel.Connectors.OpenAI 1.26.0 OpenAI-compatible providers
Microsoft.SemanticKernel.Connectors.Ollama 1.26.0-alpha Local LLM support
MongoDB.Driver 3.0.0 Database client
Swashbuckle.AspNetCore 6.6.2 Swagger/OpenAPI

ChatRAG-Specific:

Package Version Purpose
Qdrant.Client 1.14.0 Qdrant vector DB
Microsoft.Extensions.VectorData.Abstractions 9.0.0-preview Vector storage abstraction
BlazMapper 0.0.5 Object mapping

3.9 Data Flow Diagrams

ChatAPI Flow:

HTTP Request
    ↓
Classifier (determines type: Chat/Company/HR)
    ↓
ResponseFactory (selects service)
    ↓
IResponseService (delegates)
    ├─ ResponseChatService (general queries)
    ├─ ResponseCompanyService (company info)
    └─ ResponseBotRHService (HR)
    ↓
IChatCompletionService (Kernel)
    ↓
LLM Provider (OpenChat/Ollama/GPT-4)
    ↓
Response

ChatRAG Flow:

HTTP Request (ProjectId + Question)
    ↓
Query Analysis (intent detection, strategy selection)
    ↓
Hierarchical Search (4 strategies based on complexity)
    ├─ IVectorSearchService (retrieve similar documents)
    └─ ITextEmbeddingGenerationService (generate embeddings)
    ↓
Context Building & Enrichment
    ├─ Document summarization (concurrent)
    ├─ Gap identification
    └─ Related document expansion
    ↓
Confidence Verification (multi-dimensional)
    ├─ Calculate confidence metrics
    ├─ Apply relaxation rules (intent-based)
    └─ Decide: respond or fallback
    ↓
Response Generation
    ├─ PromptConfigurationService (template selection)
    ├─ IChatCompletionService (Kernel)
    ├─ LLM Provider (Groq/Ollama)
    └─ Format response with metadata
    ↓
HTTP Response

3.10 Configuration Management

Provider Selection (appsettings.json):

{
  "LLM": {
    "Provider": "Groq",
    "Model": "llama-3.1-8b-instant"
  },
  "Embeddings": {
    "Provider": "Ollama",
    "Model": "all-minilm"
  },
  "VectorDatabase": {
    "Provider": "Hybrid",
    "Qdrant": { ... },
    "MongoDB": { ... },
    "Chroma": { ... }
  }
}

4. Summary: Objects for Modification (Per CLAUDE.md Instructions)

Following your instruction "não quero que você mude nada, mas que aponte quais objetos terão que ser alterados", here are the objects that would need modification for various VideoStudy.app implementation scenarios:

4.1 If Integrating YTExtractor Services

Objects to Modify/Create:

  • YoutubeService interface - Extract interface from YTExtractor.YoutubeService
  • YoutubeDataService interface - Alternative implementation
  • ConvertTranscriptService - Adapt for VideoStudy domain
  • IVideoRepository - MongoDB access for cached videos
  • API Endpoint: POST /api/videos/analyze - Accept video URL + analysis parameters
  • VideoAnalysisRequest model - Input model with URL, language, analysis type
  • VideoAnalysisResponse model - Output model with metadata + transcript

4.2 If Integrating CSS/Design from vcart.me.novo

Objects to Modify/Create:

  • Copy CSS framework structure (Bootstrap 5.3.2)
  • Create site.css - Global application styles
  • Create video-player.css - Video display styles
  • Create analysis-panel.css - Analysis results styles
  • Reuse color palette (Bootstrap blue + purple accents)
  • Reuse component patterns (buttons, cards, forms)
  • Reuse typography system (responsive, semantic)

4.3 If Integrating Semantic Kernel (ChatRAG Pattern)

Objects to Modify/Create:

  • IAnalysisService interface - Semantic Kernel-based analysis
  • VideoAnalysisService - RAG service for transcript analysis
  • DocumentIngestionService - Adapt from ChatRAG for video metadata
  • PromptService - Template management for different analysis types
  • EmbeddingService - Text embeddings for similarity search
  • IVectorSearchService - Vector database abstraction
  • API Endpoint: POST /api/analysis/semantic - Analysis request handler
  • AnalysisResult model - Structured analysis output
  • VectorDatabaseFactory - Provider selection (Qdrant/MongoDB/Chroma)

4.4 Objects NOT to Create (Avoid Over-Engineering)

  • Custom authentication system (use existing patterns from vcart.me.novo)
  • Caching layer beyond MongoDB (trust Kernel caching)
  • Complex middleware stack
  • Custom logging infrastructure (Serilog is sufficient)
  • Multiple response types (start with simple JSON)

Phase 1 - Minimal Viable Structure

  1. Copy YTExtractor Services → VideoStudy.API

    • YoutubeService
    • ConvertTranscriptService
    • MongoDB repository pattern
  2. Implement Minimal Semantic Kernel Integration

    • Single endpoint: /api/analysis/analyze
    • Accept: video URL
    • Return: transcript + basic analysis
    • Use Semantic Kernel 1.26.0
  3. Use vcart.me.novo Design System

    • Bootstrap 5.3.2
    • Color palette (blue + purple)
    • Responsive typography
    • Button/card styles
  4. Create Blazor Hybrid Desktop App

    • Single page with video URL input
    • Button to submit
    • Text area for results
    • Use CSS from API project

Phase 2 - RAG Implementation (Future)

  • Integrate HierarchicalRAGService pattern from ChatRAG
  • Add vector database (Qdrant or MongoDB)
  • Implement document ingestion pipeline
  • Add confidence verification

Phase 3 - Advanced Features (Future)

  • Accordion for result sections
  • Drag & drop for file uploads
  • PDF preview
  • Progress bar
  • Multiple analysis strategies

6. Questions to Answer (Per Phase Requirements)

These questions will be answered in the next phase after implementation:

  1. Semantic Kernel Version: 1.26.0 (recommended, latest stable)
  2. Stepwise Planner Alternative: Hierarchical query analysis + multi-stage execution (from ChatRAG)
  3. YTExtractor Essential Dependencies:
    • yt-dlp tool (external executable)
    • MongoDB.Driver 3.1.0
    • Google.Apis.YouTube.v3 1.69.0.3707 (optional backup)
  4. vcart.me.novo Design: Bootstrap 5.3.2 (NOT Tailwind)

Document Version History

Version Date Changes
1.0 2026-02-06 Initial comprehensive reference analysis

EOF