EntGAN: A Distributed GAN Framework Baed on Multi-task #337

nailtu30 · 2022-07-28T10:16:17Z

/kind feature
Fixes #336

kubeedge-bot · 2022-07-28T10:16:26Z

Welcome @nailtu30! It looks like this is your first PR to kubeedge/sedna 🎉

JoeyHwong-gk · 2022-07-30T01:18:09Z

/ping @MooreZheng

JoeyHwong-gk · 2022-07-30T01:19:50Z

hi @nailtu30 ,you should remove the unnecessary system files, such as .DS_Store

kubeedge-bot · 2022-08-02T02:03:16Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign sids-b after the PR has been reviewed.
You can assign the PR to them by writing /assign @sids-b in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

MooreZheng · 2022-08-02T02:30:10Z

docs/proposals/EntGAN.md

+We will explain our ideas on system architecture and learning process.
+
+### System Architecture
+As shown in the figure below, EntGAN has a set of execution modules and a set of control modules. The execution module performs GAN training tasks, including local generators and local discriminators on each worker, as well as global generator and global discriminator on the server.    


A critical weakpoint of this proposal is that the proposed architecture has not yet consider the integration of Sedna.
Especially what is the relation between the proposed modules and Sedna modules?

In the new version, we see an integration, but there are still concerns about motivation.
See: #337 (comment)

MooreZheng · 2022-08-02T02:31:59Z

docs/proposals/EntGAN.md

+
+Generative adversarial networks(GAN) has been widely used to solve the challenges of small samples and heterogeneous data. In recent years, distributed GAN has expanded and accelerated the training of GAN. The existing distributed GANs aim to train a certain class of discriminators to generate a single kind of fake data. However, in real life, there are applications that need to train many kinds of discriminators at the same time, such as image conversion and robot inspection.     
+
+Therefore, we propose an enhanced GAN framework to help sole **multi-task** and **memory usage** problem. We call the framework EntGAN.     


The motivation here is a little bit too abstract. What is the targeting scenario and dataset?

In the new version, dataset is added in the proposal. But the story is still not yet strong enough considering limited edge resources or budgets. See #337 (comment)

MooreZheng · 2022-08-08T12:12:28Z

docs/proposals/EntGAN.md

+We deploy global generator and global discriminator on Global Controller and edge generator and edge discriminator on Local Controller. After updating local dicriminator, we use sedna lifelong learning to again update and enhance its parameter.
+
+
+### The flows of GAN job


When edge resource is limited and expensive, leaving local discriminator and generator may not be cost-effective. The story itself is not strong enough. For tackling data heterogeneity, my suggestion is to integrate the proposed GAN to unseen task processing of Sedna lifelong learning, instead of proposing a new scheme.

docs/proposals/EntGAN.md

MooreZheng · 2022-08-09T07:13:23Z

The current proposal of GAN itself is fine as a algorithm, e.g., for lifelong learning. But a distributed GAN as a standalone scheme for Sedna like edge-cloud joint inference requires notrivial workloads on the module design.

My suggestion is to make the proposal as two phases. Phase One focuses on pure algorithm GAN on lifelong learning using Ianvs. Phase Two focuses on intergating GAN into Sedna.

luosiqi · 2022-10-18T03:13:30Z

Lack integration solution to Ianvs. In my opinion, the proposed GAN framework should be embeded into the training phase of Ianvs.

MooreZheng · 2022-10-18T03:13:32Z

docs/proposals/EntGAN.md

+
+In the process of Sedna lifelong learning, there would be a chance to confront unknown tasks, whose data are always heterogeneous small sample. Generate Adversarial Networks(GAN) is the start-of-art generative model and GAN can generate fake data according to the distribution of the real data. Naturally, we try to utilize GAN to handle small sample problem. Self-taught learning is an approach to improve classfication performance using sparse coding to construct higher-level features with the unlabeled data. Hence, we combine GAN and self-taught learning to help Sedna lifelong learning handle unknown tasks.
+
+### Goals


See the previous discussion in #337 (comment)

The story is not yet completed

How would we solve the small data problem in lifelong learning
0) lifelong learning limitation: lifelong learning tackles the small data issue by incrementally training with label data. But labeled data is labor-intensive and its collection is time-consuming.

The proposal reduces the time for data collection. We generate data by GAN instead of real-world data collection.

The proposal reduces the intensive labor. We leveraged self-taught learning to eliminate the labeling job.

it would be much improved with the targeting scenario and dataset added, i.e., semantic segmentation and Cityscape.

The current version does not express the limitation of the current lifelong learning.

The current lifelong learning is designed to tackle small sample problems.

Why do we still need a GAN? GAN cannot generate labeled data so far. Why not just set a camera on a car and collect more data?
The author might want to consider adding a related story mentioned in https://github.com/kubeedge/sedna/pull/337/files#r997678113.

MooreZheng · 2022-10-18T03:22:33Z

docs/proposals/EntGAN.md

+### GAN Design
+We use the networks design by [TOWARDS FASTER AND STABILIZED GAN TRAINING FOR HIGH-FIDELITY FEW-SHOT IMAGE SYNTHESIS](https://openreview.net/forum?id=1Fqg133qRaI). The design is aimed for small training data and pour computing devices. Therefore, it is perfectly suitable for handling unkwnon tasks of Sedna lifelong learning. The network is shown below [GAN Desin](images/EntGAN%20GAN.png).    
+
+![](images/EntGAN%20GAN.png)


The architecture is needed for the proposal. We see that the GAN is now put in the unseen task processing. It would be better to show the overall architecture to let the user know which scheme it belongs (i.e., lifelong learning), not only the unseen task processing component.

See previous comment: #337 (comment)

Not yet resolved

MooreZheng · 2022-10-25T02:11:54Z

docs/proposals/EntGAN.md

+1. GAN exploits the unknown task sample to generate more fake sample. 
+2. Self-taught learning unit utilize the fake sample and orginal unknown task sample and its label to train a classifier.
+3. A well trained classifier is output.
+


What are the targeting scenario and dataset?

MooreZheng · 2022-10-25T02:14:45Z

The proposal is overall good but

need to be polished to strengthen the motivation
make the scenario clear instead of a framework without any supporting dataset examples

EntGAN: A Distributed GAN Framework Baed on Multi-task

a21bc65

kubeedge-bot added the kind/feature Categorizes issue or PR as related to a new feature. label Jul 28, 2022

kubeedge-bot requested review from JimmyYang20 and TymonXie July 28, 2022 10:16

kubeedge-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jul 28, 2022

nailtu30 added 4 commits August 1, 2022 14:21

remove .DS_Store

16886aa

remove doc/.DS_Store

380718a

remove doc/proposal/.DS_Store

0f35c68

updates proposal

6229609

MooreZheng reviewed Aug 2, 2022

View reviewed changes

modify EntGAN proposal

24fa07d

MooreZheng reviewed Aug 8, 2022

View reviewed changes

JimmyYang20 reviewed Aug 9, 2022

View reviewed changes

docs/proposals/EntGAN.md Show resolved Hide resolved

nailtu30 added 2 commits August 14, 2022 16:35

modify design

4d4649c

update 0825 2022

e283aa9

MooreZheng mentioned this pull request Aug 31, 2022

a proposal for small sample problem kubeedge/ianvs#34

Closed

MooreZheng reviewed Oct 18, 2022

View reviewed changes

MooreZheng reviewed Oct 25, 2022

View reviewed changes

Merge branch 'kubeedge:main' into main

adef7b0

nailtu30 closed this by deleting the head repository Oct 31, 2022


		Generative adversarial networks(GAN) has been widely used to solve the challenges of small samples and heterogeneous data. In recent years, distributed GAN has expanded and accelerated the training of GAN. The existing distributed GANs aim to train a certain class of discriminators to generate a single kind of fake data. However, in real life, there are applications that need to train many kinds of discriminators at the same time, such as image conversion and robot inspection.

		Therefore, we propose an enhanced GAN framework to help sole multi-task and memory usage problem. We call the framework EntGAN.

		We deploy global generator and global discriminator on Global Controller and edge generator and edge discriminator on Local Controller. After updating local dicriminator, we use sedna lifelong learning to again update and enhance its parameter.


		### The flows of GAN job


		In the process of Sedna lifelong learning, there would be a chance to confront unknown tasks, whose data are always heterogeneous small sample. Generate Adversarial Networks(GAN) is the start-of-art generative model and GAN can generate fake data according to the distribution of the real data. Naturally, we try to utilize GAN to handle small sample problem. Self-taught learning is an approach to improve classfication performance using sparse coding to construct higher-level features with the unlabeled data. Hence, we combine GAN and self-taught learning to help Sedna lifelong learning handle unknown tasks.

		### Goals

EntGAN: A Distributed GAN Framework Baed on Multi-task #337

EntGAN: A Distributed GAN Framework Baed on Multi-task #337

Uh oh!

Conversation

nailtu30 commented Jul 28, 2022

Uh oh!

kubeedge-bot commented Jul 28, 2022

Uh oh!

JoeyHwong-gk commented Jul 30, 2022

Uh oh!

JoeyHwong-gk commented Jul 30, 2022

Uh oh!

kubeedge-bot commented Aug 2, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MooreZheng commented Aug 9, 2022

Uh oh!

luosiqi commented Oct 18, 2022

Uh oh!

MooreZheng Oct 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MooreZheng Oct 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MooreZheng commented Oct 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

MooreZheng Oct 18, 2022 •

edited

Loading

MooreZheng Oct 18, 2022 •

edited

Loading

MooreZheng commented Oct 25, 2022 •

edited

Loading