ADR-002: CouchDB as Primary Database

Decision Analysis and Resolution: SW360 Primary Database

Created by: SW360 Architecture Team
Original Decision: 2014
Reformatted: April 2026
Status: Accepted
Estimated read time: 12 minutes

Table of Contents

1. Background
2. Goal
3. Key Principles
4. Key Inputs, Assumptions and Restrictions
5. Options Analysis
   • Option 1 - PostgreSQL/MySQL
   • Option 2 - MongoDB
   • Option 3 - Apache CouchDB
   • Option 4 - Elasticsearch
6. Criteria for Making a Decision
7. Final Decision
8. Contributors
9. Evolution & Updates

Background

SW360 manages software components, releases, projects, and licenses for open source license compliance. The data model is characterized by:

• Complex, evolving structures: Components have different fields than licenses; new fields are frequently added
• Binary attachments: Source code archives, clearing reports, license texts
• Full-text search requirements: Search across all document fields
• Enterprise deployments: Need horizontal scaling and high availability
• Replication needs: Disaster recovery and geographic distribution

Why This Decision Matters: The database choice fundamentally affects every aspect of SW360—from development productivity to operational complexity to feature capabilities.

Goal

The goal of this decision analysis is to:

1. Select a primary database that supports SW360’s document-centric data model
2. Enable storage of binary attachments alongside metadata
3. Provide full-text search capabilities
4. Support enterprise deployment requirements (HA, replication, scaling)
5. Minimize operational complexity

Key Principles

Key Inputs, Assumptions and Restrictions

Options Analysis

Option 1 - PostgreSQL/MySQL

Summary

Use a traditional relational database with JSONB columns for flexible schema support. This leverages established SQL expertise while adding semi-structured data capabilities.

Conceptual View

┌─────────────────────────────────────────────┐
│               PostgreSQL                     │
├─────────────────────────────────────────────┤
│  components (id, type, data JSONB)          │
│  releases (id, component_id, data JSONB)    │
│  attachments (id, blob BYTEA, metadata)     │
│  ... foreign keys, indexes ...              │
└─────────────────────────────────────────────┘

Impact / Changes Required

• Design hybrid schema with JSONB for flexible fields
• Implement separate attachment storage (filesystem or blob)
• Add full-text search via PostgreSQL FTS or external Elasticsearch
• Build custom replication for HA

SWOT Analysis

Option 2 - MongoDB

Summary

Use MongoDB as a document-oriented database with native JSON storage, flexible schemas, and GridFS for large file storage.

Conceptual View

graph TB
    subgraph MongoDB
        direction TB
        C1[Collections:<br/>components, releases,<br/>projects, licenses]
        C2[GridFS: attachments]
    end

Impact / Changes Required

• Design document collections with embedded relationships
• Configure GridFS for attachment storage
• Set up replica sets for high availability
• Integrate MongoDB text search

SWOT Analysis

Option 3 - Apache CouchDB

Summary

Use Apache CouchDB, a document-oriented database with HTTP API, native attachment storage, and built-in replication capabilities.

Conceptual View

graph TB
    subgraph CouchDB["Apache CouchDB"]
        direction TB
        DB1["sw360db/<br/>component:abc123 + attachments<br/>release:def456 + attachments<br/>license:ghi789"]
        DB2["sw360users/"]
        DB3["sw360attachments/"]
        F["Features:<br/>HTTP REST API | MVCC | Replication | Attachments"]
    end

Impact / Changes Required

• Design documents by type with type discriminator field
• Attach binaries directly to documents
• Configure views (MapReduce) for queries
• Set up replication for disaster recovery

SWOT Analysis

Option 4 - Elasticsearch

Summary

Use Elasticsearch as the primary database, leveraging its distributed search and document storage capabilities.

Conceptual View

graph TB
    subgraph ES["Elasticsearch"]
        direction TB
        I1["Indices:<br/>sw360-components<br/>sw360-releases<br/>sw360-licenses"]
        I2["(attachments require external storage)"]
    end

Impact / Changes Required

• Design indices with mappings
• Implement external attachment storage
• Handle document versioning manually
• Manage cluster operations

SWOT Analysis

Criteria for Making a Decision

T-Shirt Sizing Scale

Weighted Evaluation Matrix

Score Summary

Final Decision

Selected Option: Apache CouchDB

Rationale

Apache CouchDB was selected as the primary database for SW360 based on:

1. Highest Weighted Score (683.75) - Clear winner across combined criteria

2. Attachment Storage (XL) - Only option with native document-level attachment storage:

   {
     "_id": "release:abc123",
     "_attachments": {
       "source.tar.gz": { "content_type": "application/gzip", "length": 15728640 },
       "clearing-report.pdf": { "content_type": "application/pdf", "length": 524288 }
     },
     "name": "Apache Commons",
     "version": "3.12"
   }
   

3. Schema Flexibility (XL) - Documents can have different structures:

   • Components have vendor references
   • Licenses have full text and obligations
   • No migration scripts needed for new fields

4. Built-in Replication (XL) - Master-master replication out of the box:

   curl -X POST http://localhost:5984/_replicate \
     -d '{"source": "sw360db", "target": "http://backup:5984/sw360db", "continuous": true}'
   

5. HTTP REST API (XL) - Debug with standard tools:

   # Get a document
   curl http://localhost:5984/sw360db/component:abc123
   
   # Search with Mango
   curl -X POST http://localhost:5984/sw360db/_find \
     -d '{"selector": {"type": "component", "componentType": "OSS"}}'
   

Implementation Notes

Database Structure:

CouchDB Instance
├── sw360db           # Main application data
│   ├── components    # type: "component"
│   ├── releases      # type: "release"
│   ├── projects      # type: "project"
│   ├── licenses      # type: "license"
│   └── attachments   # stored on documents
├── sw360users        # User data
└── sw360attachments  # Attachment metadata index

Access Patterns:

• By ID: Direct document retrieval
• By View: Pre-computed MapReduce indexes
• By Mango: Ad-hoc JSON queries

Review Triggers

This decision should be revisited if:

• CouchDB development stagnates or project becomes unmaintained
• Multi-document transactions become critical requirement
• Horizontal scaling beyond master-master replication needed
• Full-text search performance becomes limiting factor

Contributors

Evolution & Updates

Search Engine Changes

SDK Migration

Consequences Summary

Positive

• ✅ Flexible data model—add fields without migrations
• ✅ Attachment handling—binaries co-located with metadata
• ✅ HTTP API—simple debugging with curl/browser
• ✅ Built-in replication—easy backup and disaster recovery
• ✅ MVCC—automatic conflict detection
• ✅ Apache License—true open source

Negative

• ⚠️ No SQL joins—related data fetched separately or denormalized
• ⚠️ Eventual consistency—not suitable for strict transaction requirements
• ⚠️ Query limitations—complex queries need MapReduce views
• ⚠️ Memory usage—views can consume significant memory
• ⚠️ Learning curve—different paradigm from SQL databases

Technical Debt Created

• MapReduce views require JavaScript maintenance
• External search integration needed (CouchDB-Lucene → Nouveau)
• Ektorp → Cloudant SDK migration required (completed 2024)

Revision History