Merge pull request #1018 from gpt-engineer-org/viborc-roadmap-updates

Update ROADMAP.md with new structure

Author: viborc

ROADMAP.md

@@ -1,56 +1,92 @@
 # Roadmap
 
-<img width="800" alt="image" src="https://github.com/gpt-engineer-org/gpt-engineer/assets/4467025/1af9a457-1ed8-4ab3-90e6-4fa52bbb8c09">
-
-
-There are three main milestones we believe will greatly increase gpt-engineer's reliability and capability:
-- [x] Continuous evaluation of our progress 🎉
-- [ ] Test code and fix errors with LLMs
-- [ ] Make code generation become small, verifiable steps
-
-## Our current focus:
-
-- [x] **Continuous evaluation of progress 🎉**
-  - [x] Create a step that asks “did it run/work/perfect” in the end of each run [#240](https://github.com/gpt-engineer-org/gpt-engineer/issues/240) 🎉
-  - [x] Collect a dataset for gpt engineer to learn from, by storing code generation runs 🎉
-  - [ ] Run the benchmark multiple times, and document the results for the different "step configs" [#239](https://github.com/gpt-engineer-org/gpt-engineer/issues/239)
-  - [ ] Improve the default config based on results
-- [ ] **Self healing code**
-  - [ ] Run the generated tests
-  - [ ] Feed the results of failing tests back into LLM and ask it to fix the code
-- [ ] **Let human give feedback**
-  - [ ] Ask human for what is not working as expected in a loop, and feed it into LLM to fix the code, until the human is happy
-- [ ] **Improve existing projects**
-  - [ ] Decide on the "flow" for the CLI commands and where the project files are created
-  - [ ] Add an "improve code" command
-  - [ ] Benchmark capabilities against other tools
-
-## Experimental research
-This is not our current focus, but if you are interested in experimenting: Please
-create a thread in Discord #general and share your intentions and your findings as you
-go along. High impact examples:
-- [ ] **Make code generation become small, verifiable steps**
-  - [ ] Ask GPT4 to decide how to sequence the entire generation, and do one
-  prompt for each subcomponent
-  - [ ] For each small part, generate tests for that subpart, and do the loop of running the tests for each part, feeding
-results into GPT4, and let it edit the code until they pass
-- [ ] **Ad hoc experiments**
-  - [ ] Try Microsoft guidance, and benchmark if this helps improve performance
-  - [ ] Dynamic planning: Let gpt-engineer plan which "steps" to carry out itself, depending on the
-task, by giving it few shot example of what are usually "the right-sized steps" to carry
-out for such projects
-
-## Codebase improvements
-By improving the codebase and developer ergonomics, we accelerate progress. Some examples:
-- [ ] Set up automatic PR review for all PRs with e.g. Codium pr-agent
-- [ ] LLM tests in CI: Run super small tests with GPT3.5 in CI, that check that simple code generation still works
+<img width="800" alt="image" src="https://github.com/gpt-engineer-org/gpt-engineer/assets/48092564/06bce891-00ef-4052-bbbd-77af8b843fff">
+
+
+This document is a general roadmap guide to the gpt-engineer project's strategic direction. 
+Our goal is to continually improve by focusing on three main pillars: 
+- User Experience, 
+- Technical Features, and 
+- Performance Tracking/Testing. 
+
+Each pillar is supported by a set of epics, reflecting our major goals and initiatives.
+
+## Main Pillars and Epics
+### 👥 User Experience
+
+- **Easy installation** - We are committed to ensuring our solution is accessible, with straightforward installation and clear instructions, complemented by Docker image instructions.
+
+- **Optimized workflows** - Our focus is on optimizing human-in-the-loop workflows to enhance productivity and ease of use.
+
+- **Git integration** - We facilitate seamless version control with automatic Git integration.
+
+- **Up-to-date documentation** - Providing concise, automatically updated documentation, including a practical Cookbook to help users make the most of gpt-engineer.
+
+
+### ⚙️ Technical Features
+- **Self-healing code** - Development of advanced self-healing code features to increase resilience and improve the general quality of generated code.
+
+- **Test generation** - Implementing automatic test generation to streamline development and ensure code reliability.
+
+- **Algorithm integration** - Establishing a pipeline for integrating the latest algorithms from research, keeping us at the cutting edge.
+
+- **Startup partnerships** - We are interested in forging partnerships with startups to utilize and showcase our technical features, fostering a collaborative ecosystem.
+
+
+### 📊 Performance Tracking/Testing
+- **Test coverage in CI** - Achieving complete test coverage in Continuous Integration (CI), including compatibility with various Python versions and Docker.
+- **Benchmark suite** - Creating a comprehensive benchmark suite, including SWE-bench, APPS, and more, to measure and improve performance.
+- **Infrastructure for benchmarks** - Setting up dedicated infrastructure for periodic benchmark evaluation to ensure consistent performance.
+- **Benchmark scoreboard** - Introducing a public benchmark scoreboard that is open to other agents, including time trace for transparent and competitive performance tracking.
+
+
+## Tracking Progress with GitHub Projects
+
+We are using [GitHub Projects](https://github.com/orgs/gpt-engineer-org/projects/3) to track the progress of our roadmap.
+
+Each issue within our project is categorized under one of the main pillars and, in most cases, associated epics.
+
+The issues can be in various states:
+
+- **Backlog** - Ideas and tasks waiting to be prioritized.
+- **Todo** - Tasks that are prioritized and planned for the upcoming sprints.
+- **In Progress** - Work that is actively being developed or implemented.
+- **Done** - Completed tasks that have met our definition of done.
+- **Discarded** - Tasks or ideas that have been considered but are not proceeding.
+
+
+
+## Organizing issues and roadmap with views
+
+
+![image](https://github.com/gpt-engineer-org/gpt-engineer/assets/48092564/4be1946b-3eee-4977-9c9e-095caabeb9f1)
+
+
+The gpt-engineer project's roadmap is organized into several views, each offering a unique perspective on our work and progress. These views help the team and community to understand where things stand and what is being prioritized quickly.
+
+### General View
+
+The **[General View](https://github.com/orgs/gpt-engineer-org/projects/3/views/1)** provides a comprehensive list of all issues, epics, and their current status. In this view, you can see a list of issues, their titles, pillar categorizations, epics, status, and assignees. It's a master list from which we can access any specific issue's details and track its journey from conception to completion. This high-level view is perfect for getting a snapshot of the entire project's progress at a glance.
+
+### Kanban View
+
+The **[Kanban View](https://github.com/orgs/gpt-engineer-org/projects/3/views/2)** visualizes our workflow in a traditional Kanban board format. It is divided into columns representing the state of issues from "Backlog" to "Done," allowing us to track the work flow through our development process. We see it as a dynamic tool where tasks are moved through stages, and it provides a visual representation that helps maintain the flow of tasks and promptly address any bottlenecks.
+
+### By Pillars
+
+The **[By Pillars](https://github.com/orgs/gpt-engineer-org/projects/3/views/3)** View categorizes issues under the main pillars of User Experience, Technical Features, and Performance Tracking/Testing. This view is crucial for maintaining focus on our foundational pillars. It filters issues into groups based on the pillar they contribute to, ensuring that our development efforts are evenly distributed and aligned with our strategic goals.
+
+### By Epics
+
+The **[By Epics](https://github.com/orgs/gpt-engineer-org/projects/3/views/4)** View organizes issues according to the epics they belong to. This detailed breakdown helps us focus on the larger goals we are working towards and manage related tasks effectively. Epics are our large-scale objectives that span multiple issues. The By Epics View allows us to filter and manage tasks that collectively contribute to a significant, overarching goal, clarifying the progression of these key initiatives.
+
 
 # How you can help out
 
 You can:
 
-- Post a "design" as a google doc in our Discord and ask for feedback to address one of the items in the roadmap
+- Post a "design" as a Google Doc in our [Discord](https://discord.com/channels/1119885301872070706/1120698764445880350), and ask for feedback to address one of the items in the roadmap
 - Submit PRs to address one of the items in the roadmap
 - Do a review of someone else's PR and propose next steps (further review, merge, close)
 
-Volunteer work in any of these will get acknowledged.
+🙌 Volunteer work in any of these will get acknowledged.🙌