Skip to content

Commit 6390d84

Browse files
ShashwatDadhichShashwatDevtrongireesh-naiduprakash100198pghildiyal
authored
feat: jobs feature (#3074)
* Made chenges * Made chenges * Made chenges * Clone job function added * fixed typo * fixed typo * Merging main to job-feature-branch (#2955) * fix: Installed apps rbac optimisation (#2918) * rbac enforced in batch * minor restructuring * minor fix * code comments added * throwing unauthorised error * fix: pvc mounted on pods for cache handling (#2912) * first cut pvc for ci * isPvcMounted flag introduced * cache invalidate and global invalidate handling * cache path made configurable * pvc mounted false * PVC mounted at two new paths for storing build and buildx cache * mnor fix * removed redundant objects from struct * chenged names of env variables for paths * buildx cache path restored after doing POC --------- Co-authored-by: Prashant Ghildiyal <[email protected]> Co-authored-by: Kripansh <[email protected]> * updated log type for pipeline status methods (#2926) * docs: minor updates ingress (#2931) * minor updates ingress * minor updates ingress * finished on time update fix (#2932) Co-authored-by: Kripansh <[email protected]> * fix delete pipeline app status check (#2939) * feat: added option to propagate custom tag to k8s resources for that application (#2841) * wip * sql file renamed * propagation in labels support for create app api * validating labels if propagate is true * refactoring * sql file no. changed * removed required field from bool * sql file no. changed * removed cascade * alter app lable key * db migration seq fix for global tags --------- Co-authored-by: vikramdevtron <[email protected]> * excluding inactive ci-pipelin-material from pre-cd stage (#2920) * feat: cluster bearer token hide from dashboard (#2894) * cluster token config removed mandatory * api spec added for cluster update and create, and cluster list api changes for token * fix check config for cluster token * docs: mount pvc (#2941) * added pvc * added pvc * added pvc * added pvc * added pvc * feat: Resource browser child ref (#2913) * child rbac handling init commit * resource manifest validate handling added * handle rbac case * wire gen fix * gvk passed and ap resource handling * code cleaning * dead code cleaning * removed unused func * fix: k8s log stream cpu issue (#2929) * updated buffer size * refactored code for getting logs from k8s * updated Dockerfile * updated Dockerfile * downgraded go version * removed redundant log * sorting_changes (#2935) * support given for cloning linked ci pipelines via workflow cloning API (#2944) * docs: Added preset plugins (#2904) * added preset plugins * added preset plugins * Added semgrep, codacy * updates * updated screenshots * updates --------- Co-authored-by: Gireesh Naidu <[email protected]> Co-authored-by: Prakash Kumar <[email protected]> Co-authored-by: Prashant Ghildiyal <[email protected]> Co-authored-by: Kripansh <[email protected]> Co-authored-by: kartik-579 <[email protected]> Co-authored-by: SNe789 <[email protected]> Co-authored-by: Vikram <[email protected]> Co-authored-by: Manish Agrawal <[email protected]> Co-authored-by: vikramdevtron <[email protected]> Co-authored-by: kripanshdevtron <[email protected]> Co-authored-by: Dhananjay Sharma <[email protected]> * made changes to the queries * Update AppListingRepository.go * Update version.go * Update version.go * Update 1_insert.up.sql * Update 1_insert.up.sql * Update AppCloneService.go * Update AppCloneService.go * Uodated app listing api * Uodated app listing api restHandler * Updated app listing api restHandler * Updated app listing api service layer * Updated app listing api service layer * Updated app listing api service layer * Updated app listing api service layer * Updated app listing api service layer * Updated app listing api service layer * MAde the patch-ci-pipeline api * MAde the patch-ci-pipeline api * Reduced Code duplicacy * Corrected the code of clone job. * Corrected the code of clone job. * Updated the api specs * Updated the api specs * Updated the api specs * Updated the api specs * Updated the api specs * Copied the earlier deleted files * Copied the earlier deleted files * Update jobs.yaml * Updated the api end points and wrote the api specs. * Updated the api end points and wrote the api specs. * Updated the api end points and wrote the api specs. * Updated the api end points and wrote the api specs. * wire err fix * Wrote the sql migrate down command. * Removed the extra api endpoints. * Removed the commented code * Removed the commented code. * Update version.go * Added logger wherever required. * Fixed a bug. * Updated the api specs. * Added the user authentications * Added the user authentications * Added the user authentications * Added the user authentications * Added the user authentications * Added the user authentications * Added the user authentications for triggerCiPipeline * Added the user authentications for triggerCiPipeline * Updated the api specs and rbac checks * Updated the api specs and rbac checks * Updated the stage status api in case of jobs. * Updated the job autocomplete api. * Made the status changes * Fixed the bug of last succeeded time * undid the reverted changes * Job count added * Job count corrected * Job count bug fixed * Sorting order corrected * Sorting order DESC added * Corrected the app overview page deleted pipeline bug. * Corrected the app overview page deleted pipeline bug. * added column is_artifact_uploaded to the ci_artifact table. * Added the check of deleted pipeline. * Added the check of deleted pipeline. * Added the check of deleted pipeline. * Clone app * Multiple git repo patch * removed the commented lines. * Minor bug fixes * Delete Git material bug fix. * Fixed minor bug * Wrote the sql down command. * minor fix * resolved the comments of PR. * fixed a minor bug * Changed from app_store to app_type * Changed from app_store to app_type * changes as mentioned in the PR * Checked the ci_artifacts * Changes done as mentioned in the PR * Changes done as mentioned in the PR * data optimization added * Fixed minor bug * Fixed minor bug * Jobs sql up number update * Bug fix * Bug fix * Bug fix * Bug fix * Bug fix * Bug fix * Bug fix * Artifact logger added * Additional loggers removed * Logger No tasks configured for this job added * Removed No tasks configured for this job added * Added a validation for AppType * Added comments for validation for AppType * Added a default value to IsArtifactUploaded * Removed default value to IsArtifactUploaded * Added false to isArtifactUploaded * Removed false to isArtifactUploaded * Optimized sql up and down. * changed the sql number * Optimized the sql query. * Removed the history tables from sql down. * Removed the history tables from sql down. --------- Co-authored-by: ShashwatDevtron <[email protected]> Co-authored-by: Gireesh Naidu <[email protected]> Co-authored-by: Prakash Kumar <[email protected]> Co-authored-by: Prashant Ghildiyal <[email protected]> Co-authored-by: Kripansh <[email protected]> Co-authored-by: kartik-579 <[email protected]> Co-authored-by: SNe789 <[email protected]> Co-authored-by: Vikram <[email protected]> Co-authored-by: Manish Agrawal <[email protected]> Co-authored-by: vikramdevtron <[email protected]> Co-authored-by: kripanshdevtron <[email protected]> Co-authored-by: Dhananjay Sharma <[email protected]>
1 parent 58b5671 commit 6390d84

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+1178
-320
lines changed

Wire.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,6 +248,9 @@ func InitializeApp() (*App, error) {
248248
repository.NewAppListingRepositoryImpl,
249249
wire.Bind(new(repository.AppListingRepository), new(*repository.AppListingRepositoryImpl)),
250250

251+
router.NewJobRouterImpl,
252+
wire.Bind(new(router.JobRouter), new(*router.JobRouterImpl)),
253+
251254
pipelineConfig.NewPipelineRepositoryImpl,
252255
wire.Bind(new(pipelineConfig.PipelineRepository), new(*pipelineConfig.PipelineRepositoryImpl)),
253256
pipeline.NewPropertiesConfigServiceImpl,

api/bean/AppView.go

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ package bean
2020
import (
2121
"encoding/json"
2222
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
23+
"time"
2324
)
2425

2526
type AppContainer struct {
@@ -36,6 +37,11 @@ type AppContainerResponse struct {
3637
DeploymentGroupDTO DeploymentGroupDTO `json:"deploymentGroup,omitempty"`
3738
}
3839

40+
type JobContainerResponse struct {
41+
JobContainers []*JobContainer `json:"jobContainers"`
42+
JobCount int `json:"jobCount"`
43+
}
44+
3945
type DeploymentGroupDTO struct {
4046
Id int `json:"id"`
4147
Name string `json:"name"`
@@ -52,6 +58,36 @@ type CiMaterialDTO struct {
5258
SourceValue string `json:"value"`
5359
}
5460

61+
type JobContainer struct {
62+
JobId int `json:"jobId"`
63+
JobName string `json:"jobName""`
64+
Description string `json:"description"`
65+
JobCiPipelines []JobCIPipeline `json:"ciPipelines"'`
66+
}
67+
68+
type JobCIPipeline struct {
69+
CiPipelineId int `json:"ciPipelineId"`
70+
CiPipelineName string `json:"ciPipelineName"`
71+
Status string `json:"status"`
72+
LastRunAt time.Time `json:"lastRunAt"`
73+
LastSuccessAt time.Time `json:"lastSuccessAt"`
74+
}
75+
76+
type JobListingContainer struct {
77+
JobId int `json:"job_id"`
78+
JobName string `json:"job_name"`
79+
Description string `json:"description"`
80+
CiPipelineID int `json:"ci_pipeline_id"`
81+
CiPipelineName string `json:"ci_pipeline_name"`
82+
Status string `json:"status"`
83+
StartedOn time.Time `json:"started_on"`
84+
}
85+
86+
type CiPipelineLastSucceededTime struct {
87+
CiPipelineID int `json:"ci_pipeline_id"`
88+
LastSucceededOn time.Time `json:"last_succeeded_on"`
89+
}
90+
5591
type AppEnvironmentContainer struct {
5692
AppId int `json:"appId"`
5793
AppName string `json:"appName"`

api/restHandler/AppListingRestHandler.go

Lines changed: 89 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ import (
3030
"github.com/devtron-labs/devtron/client/argocdServer/application"
3131
"github.com/devtron-labs/devtron/client/cron"
3232
"github.com/devtron-labs/devtron/internal/constants"
33+
"github.com/devtron-labs/devtron/internal/sql/repository/helper"
3334
"github.com/devtron-labs/devtron/internal/sql/repository/pipelineConfig"
3435
"github.com/devtron-labs/devtron/internal/util"
3536
"github.com/devtron-labs/devtron/pkg/app"
@@ -57,6 +58,8 @@ import (
5758

5859
type AppListingRestHandler interface {
5960
FetchAppsByEnvironment(w http.ResponseWriter, r *http.Request)
61+
FetchJobs(w http.ResponseWriter, r *http.Request)
62+
FetchJobOverviewCiPipelines(w http.ResponseWriter, r *http.Request)
6063
FetchAppDetails(w http.ResponseWriter, r *http.Request)
6164
FetchAllDevtronManagedApps(w http.ResponseWriter, r *http.Request)
6265
FetchAppTriggerView(w http.ResponseWriter, r *http.Request)
@@ -157,6 +160,91 @@ func (handler AppListingRestHandlerImpl) FetchAllDevtronManagedApps(w http.Respo
157160
res, err := handler.appListingService.FetchAllDevtronManagedApps()
158161
common.WriteJsonResp(w, err, res, http.StatusOK)
159162
}
163+
func (handler AppListingRestHandlerImpl) FetchJobs(w http.ResponseWriter, r *http.Request) {
164+
userId, err := handler.userService.GetLoggedInUser(r)
165+
if userId == 0 || err != nil {
166+
handler.logger.Errorw("request err, userId", "err", err, "payload", userId)
167+
common.WriteJsonResp(w, err, "Unauthorized User", http.StatusUnauthorized)
168+
return
169+
}
170+
isSuperAdmin, err := handler.userService.IsSuperAdmin(int(userId))
171+
if !isSuperAdmin || err != nil {
172+
if err != nil {
173+
handler.logger.Errorw("request err, CheckSuperAdmin", "err", isSuperAdmin, "isSuperAdmin", isSuperAdmin)
174+
}
175+
common.WriteJsonResp(w, err, "Unauthorized User", http.StatusUnauthorized)
176+
return
177+
}
178+
var fetchJobListingRequest app.FetchAppListingRequest
179+
decoder := json.NewDecoder(r.Body)
180+
err = decoder.Decode(&fetchJobListingRequest)
181+
if err != nil {
182+
handler.logger.Errorw("request err, FetchAppsByEnvironment", "err", err, "payload", fetchJobListingRequest)
183+
common.WriteJsonResp(w, err, nil, http.StatusBadRequest)
184+
return
185+
}
186+
jobs, err := handler.appListingService.FetchJobs(fetchJobListingRequest)
187+
if err != nil {
188+
handler.logger.Errorw("service err, FetchJobs", "err", err, "payload", fetchJobListingRequest)
189+
common.WriteJsonResp(w, err, "", http.StatusInternalServerError)
190+
return
191+
}
192+
jobsCount := len(jobs)
193+
offset := fetchJobListingRequest.Offset
194+
limit := fetchJobListingRequest.Size
195+
196+
if limit > 0 {
197+
if offset+limit <= len(jobs) {
198+
jobs = jobs[offset : offset+limit]
199+
} else {
200+
jobs = jobs[offset:]
201+
}
202+
}
203+
jobContainerResponse := bean.JobContainerResponse{
204+
JobContainers: jobs,
205+
JobCount: jobsCount,
206+
}
207+
208+
common.WriteJsonResp(w, err, jobContainerResponse, http.StatusOK)
209+
}
210+
func (handler AppListingRestHandlerImpl) FetchJobOverviewCiPipelines(w http.ResponseWriter, r *http.Request) {
211+
userId, err := handler.userService.GetLoggedInUser(r)
212+
if userId == 0 || err != nil {
213+
handler.logger.Errorw("request err, userId", "err", err, "payload", userId)
214+
common.WriteJsonResp(w, err, "Unauthorized User", http.StatusUnauthorized)
215+
return
216+
}
217+
isSuperAdmin, err := handler.userService.IsSuperAdmin(int(userId))
218+
if !isSuperAdmin || err != nil {
219+
if err != nil {
220+
handler.logger.Errorw("request err, CheckSuperAdmin", "err", isSuperAdmin, "isSuperAdmin", isSuperAdmin)
221+
}
222+
common.WriteJsonResp(w, err, "Unauthorized User", http.StatusUnauthorized)
223+
return
224+
}
225+
vars := mux.Vars(r)
226+
jobId, err := strconv.Atoi(vars["jobId"])
227+
if err != nil {
228+
handler.logger.Errorw("request err, GetAppMetaInfo", "err", err, "jobId", jobId)
229+
common.WriteJsonResp(w, err, nil, http.StatusBadRequest)
230+
return
231+
}
232+
job, err := handler.pipeline.GetApp(jobId)
233+
if err != nil || job == nil || job.AppType != helper.Job {
234+
handler.logger.Errorw("Job with the given Id does not exist", "err", err, "jobId", jobId)
235+
common.WriteJsonResp(w, err, "Job with the given Id does not exist", http.StatusBadRequest)
236+
return
237+
}
238+
239+
jobCi, err := handler.appListingService.FetchOverviewCiPipelines(jobId)
240+
if err != nil {
241+
handler.logger.Errorw("request err, GetJobCi", "err", jobCi, "jobCi", jobCi)
242+
common.WriteJsonResp(w, err, nil, http.StatusBadRequest)
243+
return
244+
}
245+
246+
common.WriteJsonResp(w, err, jobCi, http.StatusOK)
247+
}
160248
func (handler AppListingRestHandlerImpl) FetchAppsByEnvironment(w http.ResponseWriter, r *http.Request) {
161249
//Allow CORS here By * or specific origin
162250
setupResponse(&w, r)
@@ -571,7 +659,7 @@ func (handler AppListingRestHandlerImpl) FetchAppStageStatus(w http.ResponseWrit
571659
}
572660
//RBAC enforcer Ends
573661

574-
triggerView, err := handler.appListingService.FetchAppStageStatus(appId)
662+
triggerView, err := handler.appListingService.FetchAppStageStatus(appId, int(app.AppType))
575663
if err != nil {
576664
handler.logger.Errorw("service err, FetchAppStageStatus", "err", err, "appId", appId)
577665
common.WriteJsonResp(w, err, nil, http.StatusInternalServerError)

api/restHandler/app/AutoCompleteRestHandler.go

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ package app
22

33
import (
44
"github.com/devtron-labs/devtron/api/restHandler/common"
5+
"github.com/devtron-labs/devtron/internal/sql/repository/helper"
56
"github.com/devtron-labs/devtron/pkg/pipeline"
67
"github.com/devtron-labs/devtron/pkg/user/casbin"
78
"github.com/gorilla/mux"
@@ -32,10 +33,20 @@ func (handler PipelineConfigRestHandlerImpl) GetAppListForAutocomplete(w http.Re
3233
v := r.URL.Query()
3334
teamId := v.Get("teamId")
3435
appName := v.Get("appName")
36+
appTypeParam := v.Get("appType")
37+
var appType int
38+
if appTypeParam != "" {
39+
appType, err = strconv.Atoi(appTypeParam)
40+
if err != nil {
41+
handler.Logger.Errorw("service err, GetAppListForAutocomplete", "err", err, "teamId", teamId, "appTypeParam", appTypeParam)
42+
common.WriteJsonResp(w, err, "Failed to parse appType param", http.StatusInternalServerError)
43+
return
44+
}
45+
}
3546
handler.Logger.Infow("request payload, GetAppListForAutocomplete", "teamId", teamId)
3647
var apps []*pipeline.AppBean
3748
if len(teamId) == 0 {
38-
apps, err = handler.pipelineBuilder.FindAllMatchesByAppName(appName)
49+
apps, err = handler.pipelineBuilder.FindAllMatchesByAppName(appName, helper.AppType(appType))
3950
if err != nil {
4051
handler.Logger.Errorw("service err, GetAppListForAutocomplete", "err", err, "teamId", teamId)
4152
common.WriteJsonResp(w, err, nil, http.StatusInternalServerError)

0 commit comments

Comments
 (0)