Incomplete/Missing Content in Retrieval for Android Project - Embedder Configuration Recommendations

[CiaShangLin]

Current Configuration

I'm currently using the following embedder configuration for a medium-sized Android project:

{
  "embedder": {
    "client_class": "OpenAIClient",
    "batch_size": 500,
    "model_kwargs": {
      "model": "text-embedding-3-small",
      "dimensions": 256,
      "encoding_format": "float"
    }
  },
  "retriever": {
    "top_k": 20
  },
  "text_splitter": {
    "split_by": "word",
    "chunk_size": 350,
    "chunk_overlap": 100
  }
}

LLM Model: GPT-4o

Problem Description

When reading through my private Android project using this configuration, the parsing and retrieval appears to be missing significant content. The responses seem incomplete and lack important context from the codebase.

Questions

1. Embedding Dimensions: Is 256 dimensions too low for code embeddings? Should I increase this for better semantic understanding of Android code?

2. Chunk Size: Is chunk_size: 350 words appropriate for Android projects (Java/Kotlin files with classes, methods, XML layouts, etc.)? Should this be adjusted?

3. Retriever Settings: Is top_k: 20 sufficient, or should it be increased for better context coverage?

4. Model Selection: Would switching to text-embedding-3-large provide better results for code understanding?

5. Android-Specific Considerations: Are there recommended configurations specifically optimized for Android projects that better handle:

   • Java/Kotlin code
   • XML layouts and resources
   • Gradle build files
   • AndroidManifest.xml
   • Multi-module project structure

Expected Behavior

The embedder should capture and retrieve comprehensive context from the Android codebase, including:

• Class implementations and their relationships
• Method implementations and logic
• Resource files (layouts, strings, etc.)
• Build configuration
• Project architecture and dependencies

Environment

• Project Type: Medium-sized Android project
• Primary Languages: Java/Kotlin
• Project Structure: Multi-module (assumed)
• Current Model: GPT-4o with OpenAI text-embedding-3-small

Requested Recommendations

What embedder configuration would you recommend for optimal performance with Android projects? Specifically:

• Optimal dimensions value
• Recommended chunk_size and chunk_overlap
• Appropriate top_k value
• Alternative embedding models if applicable
• Any Android-specific tuning parameters

Thank you!

[gerivdb]

🚀 Propositon d'intégration A2A Hybrid Search x DeepWiki (ECOS CLI powered)

Résumé

Architecture hybride 4-tiers combinant :

• VSCode API (symbol) — gitlab support #1 rapidité et pertinence
• Ripgrep (pattern) — Just using docker-compose doesn't work - macbook arm64 #2 pre-filtrage ultra-rapide
• AST & Git Graph (Cline-like) — Added openai as dependency well otherwise will not start #3 dependencies contextuelles
• SQLite-vec (fallback semantic) — 500 server error when trying to generate docs #4; offline, incrémental, multi-repos (100% local, 0 coût)

Points critiques vis-à-vis de DeepWiki (actuel) :

• Persistance : 1 .db unifié, indexation hash-based incrémentale (vs pickle full reload)
• RAM : lazy load (vs full-load pkl)
• Query 95% resolues tiers 1-3, 5% embedding fallback
• Multi-repos ECOSYSTEM-1 natif
• Intégrable comme backend DeepWiki (adapter rag.py → a2a_hybrid_search); ou DeepWiki comme frontend documentation.
• Scripting full ECOS CLI : index build, update, recherche + doc gen

Commandes ECOS CLI proposées

# Setup complet architecture
ecos a2a docs setup --search-backend hybrid --wiki-engine deepwiki
# Indexation initiale 22 repos
ecos a2a index build --repos all --incremental --workers 8
# Watch mode pour update Git
ecos a2a index watch --repos all --daemon

# Génération wiki interop DeepWiki
ecos a2a docs wiki generate --repos ECOYSTEM,BRAIN,PITCH-1

# Recherche unifiée code+docs
ecos a2a search "coordination multi-agents" --sources code,wiki

# Statistiques utilisation
ecos a2a search-stats --last-week

Gains attendus (vs FAISS pickle DeepWiki)

• 10x moins d'attente au reboot
• 98% fichiers skip via hash (update <10s au lieu de 30 min)
• Zéro coût cloud/Milvus/Qdrant/FAISS
• Z600/portable compatibilité offline

Actions immédiates

• Intégrer tiered hybrid search à la couche 4 (issue Update ws support https #269)
• Isoler DeepWiki adapter (a2a_middleware/4-docs-layer/deepwiki_adapter.py)
• Proposer PR DeepWiki pour support hash-based, lazy + search cascade hybrid
• Documentation et guides ECOS CLI (README, notebooks, tuto vidéo)

Références :

• Issue #269 (ECOSYSTEM A2A)
• DeepWiki core
• Problème embeddings lents DeepWiki
• Pourquoi Cline est rapide (blog)