Backend Implementation

This document provides a detailed technical overview of Qwello's backend implementation, focusing on the NestJS architecture, module structure, API endpoints, database integration, and job queue system.

NestJS Architecture

Qwello's backend is built on NestJS, a progressive Node.js framework that provides a robust foundation for building scalable server-side applications.

Project Structure

The backend codebase follows a modular structure organized by feature and responsibility:

src/
├── main.ts                # Application entry point
├── app.module.ts          # Root application module
├── app.controller.ts      # Root application controller
├── app.service.ts         # Root application service
├── common/               # Common utilities and helpers
│   ├── controllers/      # Common controller decorators
│   ├── db/               # Database utilities
│   ├── dto/              # Data transfer objects
│   ├── enum/             # Enumerations
│   ├── filters/          # Exception filters
│   ├── interceptors/     # Request/response interceptors
│   └── resource/         # Resource classes
├── config/               # Configuration management
├── interfaces/           # TypeScript interfaces
├── modules/              # Feature modules
│   ├── ai/               # AI integration module
│   ├── auth/             # Authentication module
│   ├── chats/            # Chat functionality
│   ├── pdf/              # PDF processing module
│   ├── search/           # Search functionality
│   └── user/             # User management
└── telegram/             # Telegram bot integration

Module Architecture

NestJS uses a modular architecture that promotes code organization, reusability, and maintainability. Each module encapsulates a specific feature or domain of the application.

// app.module.ts
import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { MongooseModule } from '@nestjs/mongoose';
import { BullModule } from '@nestjs/bull';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { AuthModule } from './modules/auth/auth.module';
import { UserModule } from './modules/user/user.module';
import { PdfModule } from './modules/pdf/pdf.module';
import { AiModule } from './modules/ai/ai.module';
import { SearchModule } from './modules/search/search.module';
import { ChatsModule } from './modules/chats/chats.module';
import { TelegramModule } from './telegram/bot.module';
import authConfig from './config/auth.config';

@Module({
  imports: [
    // Configuration
    ConfigModule.forRoot({
      isGlobal: true,
      load: [authConfig],
    }),
    
    // Database
    MongooseModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: async (configService: ConfigService) => ({
        uri: configService.get<string>('MONGODB_URI'),
        dbName: configService.get<string>('MONGODB_DB'),
      }),
    }),
    
    // Queue
    BullModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: async (configService: ConfigService) => ({
        redis: {
          host: configService.get<string>('REDIS_HOST'),
          port: configService.get<number>('REDIS_PORT'),
        },
      }),
    }),
    
    // Feature modules
    AuthModule,
    UserModule,
    PdfModule,
    AiModule,
    SearchModule,
    ChatsModule,
    TelegramModule,
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Dependency Injection

NestJS uses a powerful dependency injection system that makes it easy to manage dependencies and promote testability:

// pdf.service.ts
import { Injectable } from '@nestjs/common';
import { InjectModel } from '@nestjs/mongoose';
import { Model } from 'mongoose';
import { PdfDocument } from './entities/pdf.entity';
import { AiService } from '../ai/services/ai.service';
import { StorageService } from '../storage/storage.service';

@Injectable()
export class PdfService {
  constructor(
    @InjectModel(PdfDocument.name)
    private readonly pdfModel: Model<PdfDocument>,
    private readonly aiService: AiService,
    private readonly storageService: StorageService,
  ) {}
  
  // Service methods
}

Module Structure

Each feature module in Qwello's backend follows a consistent structure to promote maintainability and separation of concerns.

PDF Module

The PDF module handles document processing, from upload to knowledge graph generation:

// pdf.module.ts
import { Module } from '@nestjs/common';
import { MongooseModule } from '@nestjs/mongoose';
import { BullModule } from '@nestjs/bull';
import { PdfController } from './controllers/pdf.controller';
import { PdfService } from './services/pdf.service';
import { PdfProcessor } from './processors/pdf.processor';
import { PdfDocument, PdfSchema } from './entities/pdf.entity';
import { PdfRepository } from './repositories/pdf.repository';
import { AiModule } from '../ai/ai.module';

@Module({
  imports: [
    MongooseModule.forFeature([
      { name: PdfDocument.name, schema: PdfSchema },
    ]),
    BullModule.registerQueue({
      name: 'pdf',
    }),
    AiModule,
  ],
  controllers: [PdfController],
  providers: [PdfService, PdfProcessor, PdfRepository],
  exports: [PdfService],
})
export class PdfModule {}

Entity Definition

Entities define the data structure for MongoDB documents:

// pdf.entity.ts
import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';
import { Document, Schema as MongooseSchema } from 'mongoose';
import { User } from '../../user/entities/user.entity';

export enum PdfStatus {
  PENDING = 'pending',
  PROCESSING = 'processing',
  COMPLETED = 'completed',
  FAILED = 'failed',
}

@Schema({ timestamps: true })
export class PdfDocument extends Document {
  @Prop({ required: true })
  filename: string;

  @Prop()
  originalName: string;

  @Prop()
  mimeType: string;

  @Prop()
  size: number;

  @Prop()
  path: string;

  @Prop({ type: MongooseSchema.Types.ObjectId, ref: 'User' })
  user: User;

  @Prop({ default: PdfStatus.PENDING })
  status: PdfStatus;

  @Prop({ type: Object })
  metadata: Record<string, any>;

  @Prop({ type: Object })
  knowledgeGraph: Record<string, any>;

  @Prop()
  error: string;

  @Prop()
  createdAt: Date;

  @Prop()
  updatedAt: Date;
}

export const PdfSchema = SchemaFactory.createForClass(PdfDocument);

Repository Pattern

Repositories encapsulate database operations:

// pdf.repository.ts
import { Injectable } from '@nestjs/common';
import { InjectModel } from '@nestjs/mongoose';
import { Model } from 'mongoose';
import { PdfDocument, PdfStatus } from '../entities/pdf.entity';
import { User } from '../../user/entities/user.entity';
import { CreatePdfDto } from '../dto/create-pdf.dto';

@Injectable()
export class PdfRepository {
  constructor(
    @InjectModel(PdfDocument.name)
    private readonly pdfModel: Model<PdfDocument>,
  ) {}

  async create(createPdfDto: CreatePdfDto, user: User): Promise<PdfDocument> {
    const pdf = new this.pdfModel({
      ...createPdfDto,
      user: user._id,
    });
    return pdf.save();
  }

  async findById(id: string): Promise<PdfDocument> {
    return this.pdfModel.findById(id).exec();
  }

  async findByUser(userId: string): Promise<PdfDocument[]> {
    return this.pdfModel.find({ user: userId }).sort({ createdAt: -1 }).exec();
  }

  async updateStatus(id: string, status: PdfStatus): Promise<PdfDocument> {
    return this.pdfModel
      .findByIdAndUpdate(id, { status }, { new: true })
      .exec();
  }

  async updateKnowledgeGraph(
    id: string,
    knowledgeGraph: Record<string, any>,
  ): Promise<PdfDocument> {
    return this.pdfModel
      .findByIdAndUpdate(
        id,
        { knowledgeGraph, status: PdfStatus.COMPLETED },
        { new: true },
      )
      .exec();
  }
}

Service Layer

Services implement business logic:

// pdf.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { InjectQueue } from '@nestjs/bull';
import { Queue } from 'bull';
import { PdfRepository } from '../repositories/pdf.repository';
import { User } from '../../user/entities/user.entity';
import { CreatePdfDto } from '../dto/create-pdf.dto';
import { PdfDocument, PdfStatus } from '../entities/pdf.entity';

@Injectable()
export class PdfService {
  private readonly logger = new Logger(PdfService.name);

  constructor(
    private readonly pdfRepository: PdfRepository,
    @InjectQueue('pdf') private readonly pdfQueue: Queue,
  ) {}

  async processPdf(
    createPdfDto: CreatePdfDto,
    user: User,
  ): Promise<PdfDocument> {
    // Create PDF document in database
    const pdf = await this.pdfRepository.create(createPdfDto, user);

    // Add job to queue
    await this.pdfQueue.add('process', {
      pdfId: pdf._id.toString(),
    });

    return pdf;
  }

  async getPdfById(id: string): Promise<PdfDocument> {
    return this.pdfRepository.findById(id);
  }

  async getPdfsByUser(userId: string): Promise<PdfDocument[]> {
    return this.pdfRepository.findByUser(userId);
  }

  async updatePdfStatus(id: string, status: PdfStatus): Promise<PdfDocument> {
    return this.pdfRepository.updateStatus(id, status);
  }

  async updateKnowledgeGraph(
    id: string,
    knowledgeGraph: Record<string, any>,
  ): Promise<PdfDocument> {
    return this.pdfRepository.updateKnowledgeGraph(id, knowledgeGraph);
  }
}

Controller Layer

Controllers handle HTTP requests and define API endpoints:

// pdf.controller.ts
import {
  Controller,
  Get,
  Post,
  Param,
  UseInterceptors,
  UploadedFile,
  UseGuards,
  Req,
} from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { JwtAuthGuard } from '../../auth/guards/jwt-auth.guard';
import { PdfService } from '../services/pdf.service';
import { CreatePdfDto } from '../dto/create-pdf.dto';
import { ApiController } from '../../common/controllers/api-controller.decorator';
import { ApiVersion } from '../../common/enum/api-version.enum';

@ApiController('pdf', ApiVersion.V1)
@UseGuards(JwtAuthGuard)
export class PdfController {
  constructor(private readonly pdfService: PdfService) {}

  @Post('upload')
  @UseInterceptors(FileInterceptor('file'))
  async uploadPdf(@UploadedFile() file: Express.Multer.File, @Req() req: any) {
    const createPdfDto: CreatePdfDto = {
      filename: file.filename,
      originalName: file.originalname,
      mimeType: file.mimetype,
      size: file.size,
      path: file.path,
    };

    return this.pdfService.processPdf(createPdfDto, req.user);
  }

  @Get()
  async getPdfs(@Req() req: any) {
    return this.pdfService.getPdfsByUser(req.user._id);
  }

  @Get(':id')
  async getPdf(@Param('id') id: string) {
    return this.pdfService.getPdfById(id);
  }

  @Get(':id/knowledge-graph')
  async getKnowledgeGraph(@Param('id') id: string) {
    const pdf = await this.pdfService.getPdfById(id);
    return pdf.knowledgeGraph;
  }
}

AI Module

The AI module handles integration with AI models:

// ai.module.ts
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { AiService } from './services/ai.service';
import { CloudflareAIProvider } from './ai-providers/cloudflare-ai.provider';
import { ProviderResolver } from './services/provider-resolver.service';
import aiConfig from './ai.config';

@Module({
  imports: [
    ConfigModule.forFeature(aiConfig),
  ],
  providers: [
    AiService,
    CloudflareAIProvider,
    ProviderResolver,
  ],
  exports: [AiService],
})
export class AiModule {}

AI Service

The AI service orchestrates interactions with AI models:

// ai.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { ProviderResolver } from './provider-resolver.service';
import { ModelRequest } from '../types';

@Injectable()
export class AiService {
  private readonly logger = new Logger(AiService.name);

  constructor(protected readonly providerResolver: ProviderResolver) {}

  async request(
    request: ModelRequest,
    fallback?: ModelRequest,
  ): Promise<string> {
    try {
      return await this.providerResolver.default().query(request, fallback);
    } catch (error) {
      this.logger.error(`AI request failed: ${error.message}`);
      throw error;
    }
  }

  async processPdf(fileBuffer: Buffer): Promise<Record<string, any>> {
    // Implementation details for PDF processing
    // This would involve multiple AI requests for different stages
    // of the PDF processing pipeline
    return {};
  }
}

API Endpoints

Qwello's backend exposes a RESTful API for frontend communication.

API Controller Decorator

A custom decorator is used to standardize API controllers:

// api-controller.decorator.ts
import { Controller, applyDecorators } from '@nestjs/common';
import { ApiTags } from '@nestjs/swagger';
import { ApiVersion } from '../enum/api-version.enum';

export function ApiController(resource: string, version: ApiVersion) {
  const prefix = `api/${version}/${resource}`;
  
  return applyDecorators(
    Controller(prefix),
    ApiTags(resource),
  );
}

Authentication Endpoints

// auth.controller.ts
import { Controller, Post, Body, UseGuards, Req } from '@nestjs/common';
import { AuthService } from '../auth.service';
import { LocalAuthGuard } from '../guards/local-auth.guard';
import { JwtAuthGuard } from '../guards/jwt-auth.guard';
import { ApiController } from '../../common/controllers/api-controller.decorator';
import { ApiVersion } from '../../common/enum/api-version.enum';

@ApiController('auth', ApiVersion.V1)
export class AuthController {
  constructor(private readonly authService: AuthService) {}

  @Post('login')
  @UseGuards(LocalAuthGuard)
  async login(@Req() req: any) {
    return this.authService.login(req.user);
  }

  @Post('register')
  async register(@Body() registerDto: RegisterDto) {
    return this.authService.register(registerDto);
  }

  @Post('refresh')
  @UseGuards(JwtAuthGuard)
  async refresh(@Req() req: any) {
    return this.authService.refresh(req.user);
  }
}

PDF Endpoints

// pdf.controller.ts
import {
  Controller,
  Get,
  Post,
  Param,
  UseInterceptors,
  UploadedFile,
  UseGuards,
  Req,
} from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { JwtAuthGuard } from '../../auth/guards/jwt-auth.guard';
import { PdfService } from '../services/pdf.service';
import { ApiController } from '../../common/controllers/api-controller.decorator';
import { ApiVersion } from '../../common/enum/api-version.enum';

@ApiController('pdf', ApiVersion.V1)
@UseGuards(JwtAuthGuard)
export class PdfController {
  constructor(private readonly pdfService: PdfService) {}

  @Post('upload')
  @UseInterceptors(FileInterceptor('file'))
  async uploadPdf(@UploadedFile() file: Express.Multer.File, @Req() req: any) {
    // Implementation details
  }

  @Get()
  async getPdfs(@Req() req: any) {
    return this.pdfService.getPdfsByUser(req.user._id);
  }

  @Get(':id')
  async getPdf(@Param('id') id: string) {
    return this.pdfService.getPdfById(id);
  }

  @Get(':id/knowledge-graph')
  async getKnowledgeGraph(@Param('id') id: string) {
    const pdf = await this.pdfService.getPdfById(id);
    return pdf.knowledgeGraph;
  }
}

Database Integration

Qwello uses MongoDB for data storage, integrated through Mongoose.

MongoDB Configuration

// app.module.ts (excerpt)
import { MongooseModule } from '@nestjs/mongoose';

@Module({
  imports: [
    MongooseModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: async (configService: ConfigService) => ({
        uri: configService.get<string>('MONGODB_URI'),
        dbName: configService.get<string>('MONGODB_DB'),
      }),
    }),
    // Other imports
  ],
})
export class AppModule {}

Base Repository

A base repository class provides common database operations:

// mongodb.base.repository.ts
import { Document, Model, FilterQuery, UpdateQuery } from 'mongoose';

export abstract class MongoDBBaseRepository<T extends Document> {
  constructor(protected readonly model: Model<T>) {}

  async create(createDto: any): Promise<T> {
    const entity = new this.model(createDto);
    return entity.save();
  }

  async findById(id: string): Promise<T | null> {
    return this.model.findById(id).exec();
  }

  async findOne(filter: FilterQuery<T>): Promise<T | null> {
    return this.model.findOne(filter).exec();
  }

  async findAll(filter: FilterQuery<T> = {}): Promise<T[]> {
    return this.model.find(filter).exec();
  }

  async update(id: string, updateDto: UpdateQuery<T>): Promise<T | null> {
    return this.model
      .findByIdAndUpdate(id, updateDto, { new: true })
      .exec();
  }

  async delete(id: string): Promise<T | null> {
    return this.model.findByIdAndDelete(id).exec();
  }
}

Job Queue System

Qwello uses BullMQ for job queue management, enabling reliable background processing.

Queue Configuration

// app.module.ts (excerpt)
import { BullModule } from '@nestjs/bull';

@Module({
  imports: [
    BullModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: async (configService: ConfigService) => ({
        redis: {
          host: configService.get<string>('REDIS_HOST'),
          port: configService.get<number>('REDIS_PORT'),
        },
      }),
    }),
    // Other imports
  ],
})
export class AppModule {}

Queue Registration

// pdf.module.ts (excerpt)
import { BullModule } from '@nestjs/bull';

@Module({
  imports: [
    BullModule.registerQueue({
      name: 'pdf',
    }),
    // Other imports
  ],
})
export class PdfModule {}

Job Processing

// pdf.processor.ts
import { Process, Processor } from '@nestjs/bull';
import { Logger } from '@nestjs/common';
import { Job } from 'bull';
import { PdfService } from '../services/pdf.service';
import { AiService } from '../../ai/services/ai.service';
import { PdfStatus } from '../entities/pdf.entity';

@Processor('pdf')
export class PdfProcessor {
  private readonly logger = new Logger(PdfProcessor.name);

  constructor(
    private readonly pdfService: PdfService,
    private readonly aiService: AiService,
  ) {}

  @Process('process')
  async processPdf(job: Job<{ pdfId: string }>) {
    const { pdfId } = job.data;
    this.logger.log(`Processing PDF ${pdfId}`);

    try {
      // Update status to processing
      await this.pdfService.updatePdfStatus(pdfId, PdfStatus.PROCESSING);

      // Get PDF document
      const pdf = await this.pdfService.getPdfById(pdfId);

      // Process PDF with AI service
      const knowledgeGraph = await this.aiService.processPdf(pdf.path);

      // Update knowledge graph
      await this.pdfService.updateKnowledgeGraph(pdfId, knowledgeGraph);

      this.logger.log(`PDF ${pdfId} processed successfully`);
      return { success: true };
    } catch (error) {
      this.logger.error(`Error processing PDF ${pdfId}: ${error.message}`);
      await this.pdfService.updateError(pdfId, error.message);
      return { success: false, error: error.message };
    }
  }
}

Error Handling

Qwello implements a robust error handling system to provide consistent error responses.

Exception Filters

// http-exception.filter.ts
import {
  ExceptionFilter,
  Catch,
  ArgumentsHost,
  HttpException,
  HttpStatus,
  Logger,
} from '@nestjs/common';
import { Request, Response } from 'express';

@Catch(HttpException)
export class HttpExceptionFilter implements ExceptionFilter {
  private readonly logger = new Logger(HttpExceptionFilter.name);

  catch(exception: HttpException, host: ArgumentsHost) {
    const ctx = host.switchToHttp();
    const response = ctx.getResponse<Response>();
    const request = ctx.getRequest<Request>();
    const status = exception.getStatus();
    const errorResponse = exception.getResponse();

    const error = {
      statusCode: status,
      timestamp: new Date().toISOString(),
      path: request.url,
      method: request.method,
      message:
        typeof errorResponse === 'object' && 'message' in errorResponse
          ? errorResponse.message
          : exception.message,
    };

    this.logger.error(
      `${request.method} ${request.url} ${status}`,
      JSON.stringify(error),
    );

    response.status(status).json(error);
  }
}

Middleware

Middleware functions are used for request processing, logging, and authentication:

// logger.middleware.ts
import { Injectable, NestMiddleware, Logger } from '@nestjs/common';
import { Request, Response, NextFunction } from 'express';

@Injectable()
export class LoggerMiddleware implements NestMiddleware {
  private logger = new Logger('HTTP');

  use(req: Request, res: Response, next: NextFunction) {
    const { ip, method, originalUrl } = req;
    const userAgent = req.get('user-agent') || '';

    res.on('finish', () => {
      const { statusCode } = res;
      const contentLength = res.get('content-length');

      this.logger.log(
        `${method} ${originalUrl} ${statusCode} ${contentLength} - ${userAgent} ${ip}`,
      );
    });

    next();
  }
}

Interceptors

Interceptors provide a way to intercept and transform the request and response:

// response-time.interceptor.ts
import {
  Injectable,
  NestInterceptor,
  ExecutionContext,
  CallHandler,
  Logger,
} from '@nestjs/common';
import { Observable } from 'rxjs';
import { tap } from 'rxjs/operators';

@Injectable()
export class ResponseTimeInterceptor implements NestInterceptor {
  private readonly logger = new Logger(ResponseTimeInterceptor.name);

  intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
    const now = Date.now();
    const req = context.switchToHttp().getRequest();
    const { method, url } = req;

    return next.handle().pipe(
      tap(() => {
        const responseTime = Date.now() - now;
        this.logger.log(`${method} ${url} ${responseTime}ms`);
      }),
    );
  }
}

Authentication

Qwello uses JWT for authentication:

// auth.service.ts
import { Injectable, UnauthorizedException } from '@nestjs/common';
import { JwtService } from '@nestjs/jwt';
import { UserService } from '../user/services/user.service';
import { User } from '../user/entities/user.entity';
import * as bcrypt from 'bcrypt';

@Injectable()
export class AuthService {
  constructor(
    private readonly userService: UserService,
    private readonly jwtService: JwtService,
  ) {}

  async validateUser(email: string, password: string): Promise<any> {
    const user = await this.userService.findByEmail(email);
    if (!user) {
      throw new UnauthorizedException('Invalid credentials');
    }

    const isPasswordValid = await bcrypt.compare(password, user.password);
    if (!isPasswordValid) {
      throw new UnauthorizedException('Invalid credentials');
    }

    const { password: _, ...result } = user.toObject();
    return result;
  }

  async login(user: any) {
    const payload = { email: user.email, sub: user._id };
    return {
      access_token: this.jwtService.sign(payload),
      user,
    };
  }

  async register(registerDto: any) {
    const hashedPassword = await bcrypt.hash(registerDto.password, 10);
    const user = await this.userService.create({
      ...registerDto,
      password: hashedPassword,
    });

    const { password: _, ...result } = user.toObject();
    return this.login(result);
  }

  async refresh(user: any) {
    return this.login(user);
  }
}

Configuration Management

Qwello uses a configuration system to manage environment-specific settings:

// auth.config.ts
import { registerAs } from '@nestjs/config';

export default registerAs('auth', () => ({
  jwtSecret: process.env.JWT_SECRET || 'secret',
  jwtExpiresIn: process.env.JWT_EXPIRES_IN || '1d',
}));

This comprehensive documentation covers the technical details of Qwello's backend implementation, from the NestJS architecture and module structure to API endpoints, database integration, and job queue system. The backend's modular design, with clear separation of concerns, enables efficient development, testing, and maintenance of the application.