|
1 | 1 | # Dragonfly Roadmap |
2 | 2 |
|
3 | | -The `Supernode` of `Dragonfly` was written in Java, as a Sandbox Level Project in CNCF(Cloud Native Computing Foundation), we choose to refactor `Supernode` with Golang. In the meantime, we cherish every contributor who is willing to participate in the project. |
| 3 | +This document is used for contributors to have a better understanding of project's progress. |
4 | 4 |
|
5 | | -This document is so crucial in order for contributors to have a better understanding of the progress of the project. For our project, the following versions are tentatively defined as project milestones. In order to welcome contributors to the community, I'll also detail the important features that need to be implemented in this document. |
| 5 | +For `Dragonfly`, the following versions are tentatively defined as project milestones. |
| 6 | +The following three versions are being planned currently: |
6 | 7 |
|
7 | | -We are currently planning three versions: |
| 8 | +1. v0.4.x, improved performance and stability. |
| 9 | +2. v0.5.x, ready for production usage. |
| 10 | +3. v1.0.x, GA version. |
8 | 11 |
|
9 | | -1. v0.3.0, as the transition version between Java and Go. |
10 | | -2. v0.4.0, as a preliminary version of Go implementation. |
11 | | -3. v0.5.0, as a pre-GA version of Go implementation. |
| 12 | +## v0.4.x |
12 | 13 |
|
13 | | -## v0.3.0 |
| 14 | +This version will focus on to improved the performance and stability of `Dragonfly`. Including: |
14 | 15 |
|
15 | | -This version will exist as a transition version. In this version, we will support Java version and go version in parallel. Including: |
| 16 | +* Golang SDK |
| 17 | +* Supernode HA |
| 18 | +* Merge the dfdaemon and dfget components |
| 19 | +* Decentralized distribution |
16 | 20 |
|
17 | | -* Bugfix with the Java version. |
18 | | -* Refactor with Golang with the function of beggar version. |
| 21 | +We will try to finish it before **October 31, 2019**. |
19 | 22 |
|
20 | | -NOTE: **In the meantime,The delivery of documentations is also critical.** |
| 23 | +### Golang SDK |
21 | 24 |
|
22 | | -We will try to finish it before **February 1, 2019**. |
| 25 | +`Dragonfly` needs a Golang SDK which is stored in https://github.com/dragonflyoss/Dragonfly/tree/master/client. And then we can achieve the following goals: |
23 | 26 |
|
24 | | -### Bug fix with the Java version |
| 27 | +* every caller of `Dragonfly` can take advantages of this SDK by integrating it |
| 28 | +* golang sdk encapsulates of the details of HTTP communication |
| 29 | +* it could provide scalability and portability for other system to integrate `Dragonfly`. |
25 | 30 |
|
26 | | -In this release, the project for `Go` is not fully functional or even runnable, so we will continue to fix some bugs in the Java version to serve the general public. |
| 31 | +Related issue: [#348](https://github.com/dragonflyoss/Dragonfly/issues/348) |
27 | 32 |
|
28 | | -// TODO |
| 33 | +### Supernode HA |
29 | 34 |
|
30 | | -### Refactor with Golang |
| 35 | +As for now, we can specify multiple `supenodes` for dfclient at the same time. However, `supernode` is not aware of each other. |
| 36 | +When one of the `supernode server` is unavailable, dfclient will try to register to the one of the remained `supernodes`. |
| 37 | +The `supernode` registered will download the file from source server, reschedule pieces and re-build a P2P network because it lacks the information about the task and the peer node which will cost a lot of time. |
| 38 | +So we should make multiple `supernodes` be aware of each other. |
31 | 39 |
|
32 | | -In this release, we support the implementation of a beggar version of the project for the Go version. We will implement the main logic of work flow with simplify the scheduler, CDN and some other functions. Including: |
| 40 | +Related issue: [#232](https://github.com/dragonflyoss/Dragonfly/issues/232) |
33 | 41 |
|
34 | | -* P2P Network Register. The entire registration logic needs to be implemented. The CDN module is not implemented for the time being, we download files from the source directly and saved locally if not exist temporarily. |
35 | | -* P2P Network Download. We use a simplified version of the scheduler to complete the p2p network, which works, but is fragile. |
| 42 | +### Merge the dfdaemon and dfget components |
36 | 43 |
|
37 | | -## v0.4.0 |
| 44 | +Both `dfdaemon` and `dfget` are meant for the client and packaged together into dfclient. |
| 45 | +So we can merge them into one single command to make the `Dragonfly` look simpler and easier to be deployed. |
38 | 46 |
|
39 | | -At this point, `Dragonfly` is a completely project written by `Go`, we will implement the basic functionality of the Java version and replace it. Including: |
| 47 | +Related issue: [#806](https://github.com/dragonflyoss/Dragonfly/issues/806) |
40 | 48 |
|
41 | | -* Delete Java source code and archive it. |
42 | | -* Implement the functionality basically. |
| 49 | +### Decentralized distribution |
43 | 50 |
|
44 | | -We will try to finish it before **March 15, 2019**. |
| 51 | +Now, `dfget` talks to `supernode` to discover peers information and publish its own status. |
| 52 | +It works well, however, the user has to maintain a `supernode` server and `supernode` will become the bottleneck of the cluster. |
| 53 | +So we can use a new approach where `dfget` can discover the same information without talking to `supernode` and |
| 54 | +all the clients can form a gossip cluster, where they can fire and listen for events. |
45 | 55 |
|
46 | | -### Archive Java source code |
| 56 | +Related PR: [#594](https://github.com/dragonflyoss/Dragonfly/pull/594) |
47 | 57 |
|
48 | | -In this version, the `Go` version is basically a replacement for the Java version. So we will remove the Java source code and stop maintain it, although we'll still provide an image of the latest Java version for archive. |
| 58 | +## v0.5.x |
49 | 59 |
|
50 | | -* Package the Java code. |
51 | | -* Delete the Java source code. |
| 60 | +This version will focus on enhancing the Dragonfly's features to support more features to ready for production usage. Including: |
52 | 61 |
|
53 | | -### Implement the functionality basically |
| 62 | +* Support for more deployment options |
| 63 | +* Support multi-server computing framework. |
| 64 | +* Development of operation and maintenance tools. |
54 | 65 |
|
55 | | -In this release, we've implemented most of the functionality and some advanced features of `Dragonfly` with `Go`, to the point of being aligned with the Java version. Including: |
| 66 | +We will try to finish it before **December 30, 2019**. |
56 | 67 |
|
57 | | -* Scheduler. Complete scheduling logic, including blacklist or something others. In addition, for better extensibility, we should support it as a plug-in. |
58 | | -* CDN support. Use CDN to Cache downloaded data from source to avoid downloading same files repeatedly. |
59 | | -* Dynamic downloading rate limiting. |
60 | | -* Disk GC. |
61 | | -* Preheat image. |
| 68 | +### Support for more deployment options |
62 | 69 |
|
63 | | -## v0.5.0 |
| 70 | +As a cloud native project, we should do more work to support deploy the `Dragonfly` on the kubernets platform. Including but not limited to the following list: |
64 | 71 |
|
65 | | -This is a basically stable version, reaching the pre-GA stage. Including: |
| 72 | +* Deploy `supernode` using [Helm](https://github.com/helm/helm) in Kubernetes to simplify the complexity of scaling SuperNodes in Kubernetes. |
| 73 | +* Deploy `supernode` cluster using [Operator](https://coreos.com/operators/). |
| 74 | +* Deploy `dfget & dfdaemon` using DaemonSet in Kubernetes. |
66 | 75 |
|
67 | | -1. Stability & Security enhancement. |
68 | | -2. Function Enhancement. |
| 76 | +Related issue: [#346](https://github.com/dragonflyoss/Dragonfly/issues/346) |
69 | 77 |
|
70 | | -We will try to finish it before **April 15, 2019**. |
| 78 | +### Support multi-server computing framework |
71 | 79 |
|
72 | | -### Stability & Security enhancement |
| 80 | +With the rise of ARM servers, x86 servers will not be the only choice. |
| 81 | +However, the images of the two computing frameworks are incompatible for one application. |
| 82 | +So the user will have to maintain the different versions for the same image which will bring challenges to management. |
73 | 83 |
|
74 | | -Enhance the stability of the project. |
| 84 | +Related issue: [#775](https://github.com/dragonflyoss/Dragonfly/issues/775) |
75 | 85 |
|
76 | | -* Support more configuration items. |
77 | | -* Support private container image. |
78 | | -* Support authentication in SuperNode API. |
79 | | -* Different encryption algorithm in data transmission. |
| 86 | +### Development of operation and maintenance tools |
80 | 87 |
|
81 | | -### Function Enhancement |
| 88 | +For better to use `Dragonfly`, we plan to provide more operation and maintenance tools to make it easy, including: |
82 | 89 |
|
83 | | -Add more advanced functionality on the basis of previous versions. |
| 90 | +* A tool to validate the config file. |
| 91 | +* A tool for performance testing of `Dragonfly` clusters. |
| 92 | +* A dashboard tool for `Dragonfly`. |
84 | 93 |
|
85 | | -* Support multi-registrys. |
86 | | -* Support supernode cluster. Cluster the SuperNode to decrease possibility of failure. |
87 | | -* Supernode support storage driver. |
88 | | -* Use [IPFS](https://github.com/ipfs/go-ipfs) to share block datas between SuperNodes. |
89 | | -* Highly user-customized modules. |
| 94 | +## v1.0.x |
90 | 95 |
|
91 | | -## Future |
| 96 | +This is a stable version in GA stage. And we will focus on making `Dragonfly` support more scenarios. Including: |
92 | 97 |
|
93 | | -We're going to do a lot more work on the community ecosystem, and before GA, we need to make our project not only to be useful, but also to be perfect. Including but not limited to the following list: |
94 | | - |
95 | | -* Deploy SuperNode using [Helm](https://github.com/helm/helm) in Kubernetes to simplify the complexity of scaling SuperNodes in Kubernetes. |
96 | | -* Deploy Supernode cluster using [Operator](https://coreos.com/operators/). |
97 | | -* Deploy dfget & dfdaemon using DaemonSet in Kubernetes. |
98 | | -* Integration with [Harbor](https://github.com/goharbor/harbor). |
99 | | -* Plug-In policy for CNCF projects. |
100 | | -* Integrated monitoring mechanism, the combination of [Prometheus](https://github.com/prometheus/prometheus) and [Grafana](https://github.com/grafana/grafana) may be our first choice. |
| 98 | +* Support different encryption algorithm in data transmission. |
| 99 | +* Support multiple file transfer protocols, such as ftp, etc. |
| 100 | +* Distribution images across cloud vendors. |
| 101 | +* Support publish and subscribe mechanism. |
101 | 102 | * ...... |
102 | 103 |
|
103 | | -This document give us a trunk direction, however it's not perfect, we may adjust the plan appropriately according to the urgency of demand. Finally, we plan to reach GA of the project at **May 21, 2019**, which is the time of [Kubecon Europe 2019](https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2019/). Let's do it together. |
| 104 | +This document give us a trunk direction, however it's not perfect, we may adjust the plan appropriately according to the urgency of demand. |
0 commit comments