split tests into two steps, report ok and done separately #3646

sokra · 2023-02-06T08:25:23Z

This removes some long running tests (mac and windows) from the required PR checks.

Now the CI workflow is split into two parts, Ok (required) for the required checks and Done (non-blocking) for all checks.

The PR comment will show

🟢 CI likely successful 🟢
A few longer running steps are still running, but they should not be considered as blocking.

See workflow summary for details

after the required checks are done.

Once all checks are done, it will be updated to 🟢 CI successful 🟢 or to ⚠️ CI failed ⚠️ depending on the full result

vercel · 2023-02-06T08:25:33Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
examples-basic-web	🔄 Building (Inspect)			Feb 6, 2023 at 9:55AM (UTC)

9 Ignored Deployments

Name	Status	Preview	Updated
examples-cra-web	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
examples-designsystem-docs	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
examples-kitchensink-blog	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
examples-native-web	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
examples-nonmonorepo	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
examples-svelte-web	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
examples-tailwind-web	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
examples-vite-web	⬜️ Ignored (Inspect)		Feb 6, 2023 at 9:55AM (UTC)
turbo-site	⬜️ Ignored (Inspect)	Visit Preview	Feb 6, 2023 at 9:55AM (UTC)

github-actions · 2023-02-06T08:29:31Z

⚠️ CI failed ⚠️

The following steps have failed in CI:

Turbopack Rust tests (mac/win, non-blocking)

See workflow summary for details

github-actions · 2023-02-06T08:56:39Z

Failing next.js integration test suites

Commit: d0c0f4b

Test summary

	main (`4e5c187` / v13.1.7-canary.5)	Current (`d0c0f4b` / v13.1.7-canary.5)	Diff (Failed)
Test suites	🔴 12 / 🟢 0 (Total: 12)	🔴 12 / 🟢 0 (Total: 12)	0️⃣
Test cases	🔴 40 / 🟢 8 (Total: 50)	🔴 34 / 🟢 12 (Total: 46)	🔽 6

✅ Fixed tests:

- test/e2e/404-page-router/index.test.ts 
- test/development/acceptance-app/editor-links.test.ts 
- test/e2e/app-dir/app-rendering/rendering.test.ts 
- test/e2e/404-page-router/index.test.ts 
- test/development/acceptance-app/editor-links.test.ts 
- test/e2e/app-dir/app-rendering/rendering.test.ts

❌ Newly failed tests:

- test/e2e/middleware-custom-matchers/test/index.test.ts 
- test/development/basic/node-builtins.test.ts 
- test/e2e/app-dir/app-alias/app-alias.test.ts 
- test/e2e/middleware-custom-matchers/test/index.test.ts 
- test/development/basic/node-builtins.test.ts 
- test/e2e/app-dir/app-alias/app-alias.test.ts

Full test report

test/development/acceptance-app/component-stack.test.ts

ReactRefresh > can edit a component without losing state
ReactRefresh > cyclic dependencies

Expand output

● ReactRefresh › can edit a component without losing state

Application is in inconsistent state: timeout.

  82 |           }
  83 |           if (status !== 'pending') {
> 84 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  85 |           }
  86 |
  87 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance/helpers.ts:84:19)
  at Object.<anonymous> (development/acceptance/ReactRefresh.test.ts:20:5)

● ReactRefresh › cyclic dependencies

Application is in inconsistent state: timeout.

  82 |           }
  83 |           if (status !== 'pending') {
> 84 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  85 |           }
  86 |
  87 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance/helpers.ts:84:19)
  at Object.<anonymous> (development/acceptance/ReactRefresh.test.ts:157:5)

test/e2e/middleware-custom-matchers/test/index.test.ts

Middleware custom matchers > should match source path
Middleware custom matchers > should match has header
Middleware custom matchers > should match has query
Middleware custom matchers > should match has cookie
Middleware custom matchers > should match has host
Middleware custom matchers > should match has header value
Middleware custom matchers > should match has query on client routing
Middleware custom matchers > should match has cookie on client routing

Expand output

● Middleware custom matchers › should match source path

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  46 |     it('should match source path', async () => {
  47 |       const res = await fetchViaHTTP(next.url, '/source-match')
> 48 |       expect(res.status).toBe(200)
     |                          ^
  49 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  50 |     })
  51 |

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:48:26)

● Middleware custom matchers › should match has header

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  56 |         },
  57 |       })
> 58 |       expect(res.status).toBe(200)
     |                          ^
  59 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  60 |
  61 |       const res2 = await fetchViaHTTP(next.url, '/has-match-1')

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:58:26)

● Middleware custom matchers › should match has query

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  67 |         'my-query': 'hellooo',
  68 |       })
> 69 |       expect(res.status).toBe(200)
     |                          ^
  70 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  71 |
  72 |       const res2 = await fetchViaHTTP(next.url, '/has-match-2')

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:69:26)

● Middleware custom matchers › should match has cookie

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  80 |         },
  81 |       })
> 82 |       expect(res.status).toBe(200)
     |                          ^
  83 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  84 |
  85 |       const res2 = await fetchViaHTTP(next.url, '/has-match-3', undefined, {

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:82:26)

● Middleware custom matchers › should match has host

expect(received).toBe(expected) // Object.is equality

Expected: 404
Received: 500

  94 |     itif(!isModeDeploy)('should match has host', async () => {
  95 |       const res1 = await fetchViaHTTP(next.url, '/has-match-4')
> 96 |       expect(res1.status).toBe(404)
     |                           ^
  97 |
  98 |       const res = await fetchViaHTTP(next.url, '/has-match-4', undefined, {
  99 |         headers: {

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:96:27)

● Middleware custom matchers › should match has header value

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  119 |         },
  120 |       })
> 121 |       expect(res.status).toBe(200)
      |                          ^
  122 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  123 |
  124 |       const res2 = await fetchViaHTTP(next.url, '/has-match-5', undefined, {

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:121:26)

● Middleware custom matchers › should match has query on client routing

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('#has-match-2')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10
  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:139:9)

● Middleware custom matchers › should match has cookie on client routing

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('#has-match-3')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10
  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:156:9)

test/integration/404-page-app/test/index.test.js

404 Page Support with _app > dev mode > should not show pages/404 GIP error if _app has GIP

Expand output

● 404 Page Support with _app › dev mode › should not show pages/404 GIP error if _app has GIP

expect(received).toBe(expected) // Object.is equality

Expected: 404
Received: 500

  79 |     it('should not show pages/404 GIP error if _app has GIP', async () => {
  80 |       const res = await fetchViaHTTP(appPort, '/abc')
> 81 |       expect(res.status).toBe(404)
     |                          ^
  82 |       const $ = cheerio.load(await res.text())
  83 |       expect($('#404-title').text()).toBe('Hi There')
  84 |       expect(stderr).not.toMatch(gip404Err)

  at Object.<anonymous> (integration/404-page-app/test/index.test.js:81:26)

test/development/basic/node-builtins.test.ts

Error overlay - editor links > should be possible to open files on RSC build error
Error overlay - editor links > should be possible to open files on RSC parse error
Error overlay - editor links > should be possible to open files on module not found error

Expand output

● Error overlay - editor links › should be possible to open files on RSC build error

Application is in inconsistent state: timeout.

  76 |           }
  77 |           if (status !== 'pending') {
> 78 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  79 |           }
  80 |
  81 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance-app/helpers.ts:78:19)
  at Object.<anonymous> (development/acceptance-app/editor-links.test.ts:53:7)

● Error overlay - editor links › should be possible to open files on RSC parse error

Application is in inconsistent state: timeout.

  76 |           }
  77 |           if (status !== 'pending') {
> 78 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  79 |           }
  80 |
  81 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance-app/helpers.ts:78:19)
  at Object.<anonymous> (development/acceptance-app/editor-links.test.ts:94:7)

● Error overlay - editor links › should be possible to open files on module not found error

Application is in inconsistent state: timeout.

  76 |           }
  77 |           if (status !== 'pending') {
> 78 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  79 |           }
  80 |
  81 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance-app/helpers.ts:78:19)
  at Object.<anonymous> (development/acceptance-app/editor-links.test.ts:135:7)

test/e2e/app-dir/app-alias/app-alias.test.ts

app-dir alias handling > should handle typescript paths alias correctly
app-dir alias handling > should resolve css imports from outside with src folder presented

Expand output

● app-dir alias handling › should handle typescript paths alias correctly

expect(received).toContain(expected) // indexOf

Expected substring: "click</button>"
Received string:    "Execution of get_from_source failed·
Caused by:
    0: Execution of resolve_source_request failed
    1: Execution of CombinedContentSource::get failed
    2: Execution of create_app_source failed
    3: Execution of load_next_config failed
    4: Execution of evaluate failed
    5: Execution of get_evaluate_pool failed
    6: Execution of turbo_tasks::completion::CompletionsVc::all failed
    7: Execution of turbopack_core::asset::AssetContentVc::write failed
    8: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::content failed
    9: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::code failed
   10: Execution of content_from_relative_path failed
   11: failed to canonicalize embedded file path
   12: No such file or directory (os error 2)"

  14 |     it('should handle typescript paths alias correctly', async () => {
  15 |       const html = await next.render('/button')
> 16 |       expect(html).toContain('click</button>')
     |                    ^
  17 |     })
  18 |
  19 |     it('should resolve css imports from outside with src folder presented', async () => {

  at Object.<anonymous> (e2e/app-dir/app-alias/app-alias.test.ts:16:20)

● app-dir alias handling › should resolve css imports from outside with src folder presented

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('button')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10

test/integration/broken-webpack-plugin/test/index.test.js

Handles a broken webpack plugin (precompile) > should render error correctly

Expand output

● Handles a broken webpack plugin (precompile) › should render error correctly

expect(received).toContain(expected) // indexOf

Expected substring: "Internal Server Error"
Received string:    "Execution of get_from_source failed·
Caused by:
    0: Execution of resolve_source_request failed
    1: Execution of CombinedContentSource::get failed
    2: Execution of create_app_source failed
    3: Execution of load_next_config failed
    4: Execution of evaluate failed
    5: Execution of get_evaluate_pool failed
    6: Execution of turbo_tasks::completion::CompletionsVc::all failed
    7: Execution of turbopack_core::asset::AssetContentVc::write failed
    8: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::content failed
    9: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::code failed
   10: Execution of content_from_relative_path failed
   11: failed to canonicalize embedded file path
   12: No such file or directory (os error 2)"

  34 |   it('should render error correctly', async () => {
  35 |     const text = await renderViaHTTP(appPort, '/')
> 36 |     expect(text).toContain('Internal Server Error')
     |                  ^
  37 |
  38 |     expect(stderr).toMatch('Error: oops')
  39 |   })

  at Object.<anonymous> (integration/broken-webpack-plugin/test/index.test.js:36:18)

test/development/acceptance-app/component-stack.test.ts

ReactRefresh > can edit a component without losing state
ReactRefresh > cyclic dependencies

Expand output

● ReactRefresh › can edit a component without losing state

Application is in inconsistent state: timeout.

  82 |           }
  83 |           if (status !== 'pending') {
> 84 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  85 |           }
  86 |
  87 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance/helpers.ts:84:19)
  at Object.<anonymous> (development/acceptance/ReactRefresh.test.ts:20:5)

● ReactRefresh › cyclic dependencies

Application is in inconsistent state: timeout.

  82 |           }
  83 |           if (status !== 'pending') {
> 84 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  85 |           }
  86 |
  87 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance/helpers.ts:84:19)
  at Object.<anonymous> (development/acceptance/ReactRefresh.test.ts:157:5)

test/e2e/middleware-custom-matchers/test/index.test.ts

Middleware custom matchers > should match source path
Middleware custom matchers > should match has header
Middleware custom matchers > should match has query
Middleware custom matchers > should match has cookie
Middleware custom matchers > should match has host
Middleware custom matchers > should match has header value
Middleware custom matchers > should match has query on client routing
Middleware custom matchers > should match has cookie on client routing

Expand output

● Middleware custom matchers › should match source path

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  46 |     it('should match source path', async () => {
  47 |       const res = await fetchViaHTTP(next.url, '/source-match')
> 48 |       expect(res.status).toBe(200)
     |                          ^
  49 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  50 |     })
  51 |

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:48:26)

● Middleware custom matchers › should match has header

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  56 |         },
  57 |       })
> 58 |       expect(res.status).toBe(200)
     |                          ^
  59 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  60 |
  61 |       const res2 = await fetchViaHTTP(next.url, '/has-match-1')

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:58:26)

● Middleware custom matchers › should match has query

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  67 |         'my-query': 'hellooo',
  68 |       })
> 69 |       expect(res.status).toBe(200)
     |                          ^
  70 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  71 |
  72 |       const res2 = await fetchViaHTTP(next.url, '/has-match-2')

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:69:26)

● Middleware custom matchers › should match has cookie

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  80 |         },
  81 |       })
> 82 |       expect(res.status).toBe(200)
     |                          ^
  83 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  84 |
  85 |       const res2 = await fetchViaHTTP(next.url, '/has-match-3', undefined, {

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:82:26)

● Middleware custom matchers › should match has host

expect(received).toBe(expected) // Object.is equality

Expected: 404
Received: 500

  94 |     itif(!isModeDeploy)('should match has host', async () => {
  95 |       const res1 = await fetchViaHTTP(next.url, '/has-match-4')
> 96 |       expect(res1.status).toBe(404)
     |                           ^
  97 |
  98 |       const res = await fetchViaHTTP(next.url, '/has-match-4', undefined, {
  99 |         headers: {

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:96:27)

● Middleware custom matchers › should match has header value

expect(received).toBe(expected) // Object.is equality

Expected: 200
Received: 500

  119 |         },
  120 |       })
> 121 |       expect(res.status).toBe(200)
      |                          ^
  122 |       expect(res.headers.get('x-from-middleware')).toBeDefined()
  123 |
  124 |       const res2 = await fetchViaHTTP(next.url, '/has-match-5', undefined, {

  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:121:26)

● Middleware custom matchers › should match has query on client routing

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('#has-match-2')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10
  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:139:9)

● Middleware custom matchers › should match has cookie on client routing

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('#has-match-3')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10
  at Object.<anonymous> (e2e/middleware-custom-matchers/test/index.test.ts:156:9)

test/integration/404-page-app/test/index.test.js

404 Page Support with _app > dev mode > should not show pages/404 GIP error if _app has GIP

Expand output

● 404 Page Support with _app › dev mode › should not show pages/404 GIP error if _app has GIP

expect(received).toBe(expected) // Object.is equality

Expected: 404
Received: 500

  79 |     it('should not show pages/404 GIP error if _app has GIP', async () => {
  80 |       const res = await fetchViaHTTP(appPort, '/abc')
> 81 |       expect(res.status).toBe(404)
     |                          ^
  82 |       const $ = cheerio.load(await res.text())
  83 |       expect($('#404-title').text()).toBe('Hi There')
  84 |       expect(stderr).not.toMatch(gip404Err)

  at Object.<anonymous> (integration/404-page-app/test/index.test.js:81:26)

test/development/basic/node-builtins.test.ts

Error overlay - editor links > should be possible to open files on RSC build error
Error overlay - editor links > should be possible to open files on RSC parse error
Error overlay - editor links > should be possible to open files on module not found error

Expand output

● Error overlay - editor links › should be possible to open files on RSC build error

Application is in inconsistent state: timeout.

  76 |           }
  77 |           if (status !== 'pending') {
> 78 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  79 |           }
  80 |
  81 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance-app/helpers.ts:78:19)
  at Object.<anonymous> (development/acceptance-app/editor-links.test.ts:53:7)

● Error overlay - editor links › should be possible to open files on RSC parse error

Application is in inconsistent state: timeout.

  76 |           }
  77 |           if (status !== 'pending') {
> 78 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  79 |           }
  80 |
  81 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance-app/helpers.ts:78:19)
  at Object.<anonymous> (development/acceptance-app/editor-links.test.ts:94:7)

● Error overlay - editor links › should be possible to open files on module not found error

Application is in inconsistent state: timeout.

  76 |           }
  77 |           if (status !== 'pending') {
> 78 |             throw new Error(`Application is in inconsistent state: ${status}.`)
     |                   ^
  79 |           }
  80 |
  81 |           await new Promise((resolve) => setTimeout(resolve, 30))

  at Object.patch (development/acceptance-app/helpers.ts:78:19)
  at Object.<anonymous> (development/acceptance-app/editor-links.test.ts:135:7)

test/e2e/app-dir/app-alias/app-alias.test.ts

app-dir alias handling > should handle typescript paths alias correctly
app-dir alias handling > should resolve css imports from outside with src folder presented

Expand output

● app-dir alias handling › should handle typescript paths alias correctly

expect(received).toContain(expected) // indexOf

Expected substring: "click</button>"
Received string:    "Execution of get_from_source failed·
Caused by:
    0: Execution of resolve_source_request failed
    1: Execution of CombinedContentSource::get failed
    2: Execution of create_app_source failed
    3: Execution of load_next_config failed
    4: Execution of evaluate failed
    5: Execution of get_evaluate_pool failed
    6: Execution of turbo_tasks::completion::CompletionsVc::all failed
    7: Execution of turbopack_core::asset::AssetContentVc::write failed
    8: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::content failed
    9: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::code failed
   10: Execution of content_from_relative_path failed
   11: failed to canonicalize embedded file path
   12: No such file or directory (os error 2)"

  14 |     it('should handle typescript paths alias correctly', async () => {
  15 |       const html = await next.render('/button')
> 16 |       expect(html).toContain('click</button>')
     |                    ^
  17 |     })
  18 |
  19 |     it('should resolve css imports from outside with src folder presented', async () => {

  at Object.<anonymous> (e2e/app-dir/app-alias/app-alias.test.ts:16:20)

● app-dir alias handling › should resolve css imports from outside with src folder presented

page.waitForSelector: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator('button')
============================================================

  329 |     return this.chain(() => {
  330 |       return page
> 331 |         .waitForSelector(selector, { timeout, state: 'attached' })
      |          ^
  332 |         .then(async (el) => {
  333 |           // it seems selenium waits longer and tests rely on this behavior
  334 |           // so we wait for the load event fire before returning

  at lib/browsers/playwright.ts:331:10

test/integration/broken-webpack-plugin/test/index.test.js

Handles a broken webpack plugin (precompile) > should render error correctly

Expand output

● Handles a broken webpack plugin (precompile) › should render error correctly

expect(received).toContain(expected) // indexOf

Expected substring: "Internal Server Error"
Received string:    "Execution of get_from_source failed·
Caused by:
    0: Execution of resolve_source_request failed
    1: Execution of CombinedContentSource::get failed
    2: Execution of create_app_source failed
    3: Execution of load_next_config failed
    4: Execution of evaluate failed
    5: Execution of get_evaluate_pool failed
    6: Execution of turbo_tasks::completion::CompletionsVc::all failed
    7: Execution of turbopack_core::asset::AssetContentVc::write failed
    8: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::content failed
    9: Execution of turbopack_ecmascript::chunk::EcmascriptChunkContentVc::code failed
   10: Execution of content_from_relative_path failed
   11: failed to canonicalize embedded file path
   12: No such file or directory (os error 2)"

  34 |   it('should render error correctly', async () => {
  35 |     const text = await renderViaHTTP(appPort, '/')
> 36 |     expect(text).toContain('Internal Server Error')
     |                  ^
  37 |
  38 |     expect(stderr).toMatch('Error: oops')
  39 |   })

  at Object.<anonymous> (integration/broken-webpack-plugin/test/index.test.js:36:18)

github-actions · 2023-02-06T09:48:54Z

Benchmark for `e2b894f`

Click to view benchmark

Test	Base	PR	%
bench_hmr_to_commit/Turbopack CSR/1000 modules	10.27ms ± 0.08ms	10.14ms ± 0.11ms	-1.27%
bench_hmr_to_commit/Turbopack RCC/1000 modules	10.41ms ± 0.05ms	10.40ms ± 0.10ms	-0.05%
bench_hmr_to_commit/Turbopack RSC/1000 modules	531.08ms ± 2.77ms	525.25ms ± 5.11ms	-1.10%
bench_hmr_to_commit/Turbopack SSR/1000 modules	10.32ms ± 0.12ms	10.39ms ± 0.10ms	+0.62%
bench_hmr_to_eval/Turbopack CSR/1000 modules	9139.44µs ± 92.89µs	9328.52µs ± 98.96µs	+2.07%
bench_hmr_to_eval/Turbopack RCC/1000 modules	9421.58µs ± 119.71µs	9408.65µs ± 97.55µs	-0.14%
bench_hmr_to_eval/Turbopack SSR/1000 modules	9344.07µs ± 98.18µs	9253.19µs ± 74.57µs	-0.97%
bench_hydration/Turbopack RCC/1000 modules	4385.08ms ± 10.50ms	4386.72ms ± 16.56ms	+0.04%
bench_hydration/Turbopack RSC/1000 modules	3960.55ms ± 10.84ms	4004.84ms ± 15.59ms	+1.12%
bench_hydration/Turbopack SSR/1000 modules	3783.96ms ± 20.60ms	3789.08ms ± 16.23ms	+0.14%
bench_startup/Turbopack CSR/1000 modules	2897.58ms ± 14.80ms	2871.32ms ± 13.36ms	-0.91%
bench_startup/Turbopack RCC/1000 modules	2668.06ms ± 9.34ms	2674.91ms ± 9.91ms	+0.26%
bench_startup/Turbopack RSC/1000 modules	2574.99ms ± 9.74ms	2554.86ms ± 7.91ms	-0.78%
bench_startup/Turbopack SSR/1000 modules	2143.62ms ± 3.63ms	2141.34ms ± 6.68ms	-0.11%

github-actions · 2023-02-06T11:11:33Z

Benchmark for `d0c0f4b`

Click to view benchmark

Test	Base	PR	%
bench_hmr_to_commit/Turbopack CSR/1000 modules	10.23ms ± 0.07ms	10.24ms ± 0.07ms	+0.12%
bench_hmr_to_commit/Turbopack RCC/1000 modules	10.48ms ± 0.08ms	10.51ms ± 0.11ms	+0.29%
bench_hmr_to_commit/Turbopack RSC/1000 modules	537.33ms ± 2.26ms	535.60ms ± 3.03ms	-0.32%
bench_hmr_to_commit/Turbopack SSR/1000 modules	10.33ms ± 0.09ms	10.32ms ± 0.08ms	-0.16%
bench_hmr_to_eval/Turbopack CSR/1000 modules	9264.23µs ± 113.12µs	9179.77µs ± 97.40µs	-0.91%
bench_hmr_to_eval/Turbopack RCC/1000 modules	9445.80µs ± 78.60µs	9456.98µs ± 63.60µs	+0.12%
bench_hmr_to_eval/Turbopack SSR/1000 modules	9273.88µs ± 80.40µs	9274.66µs ± 86.80µs	+0.01%
bench_hydration/Turbopack RCC/1000 modules	4476.84ms ± 11.68ms	4503.39ms ± 7.76ms	+0.59%
bench_hydration/Turbopack RSC/1000 modules	4056.75ms ± 21.35ms	4059.89ms ± 14.42ms	+0.08%
bench_hydration/Turbopack SSR/1000 modules	3911.46ms ± 14.23ms	3939.59ms ± 10.73ms	+0.72%
bench_startup/Turbopack CSR/1000 modules	2908.62ms ± 8.60ms	2914.66ms ± 10.17ms	+0.21%
bench_startup/Turbopack RCC/1000 modules	2699.19ms ± 6.25ms	2710.55ms ± 6.92ms	+0.42%
bench_startup/Turbopack RSC/1000 modules	2605.24ms ± 9.64ms	2586.96ms ± 13.11ms	-0.70%
bench_startup/Turbopack SSR/1000 modules	2222.34ms ± 3.96ms	2220.78ms ± 5.60ms	-0.07%

alexkirsz · 2023-02-06T12:52:48Z

.github/workflows/test.yml

+          subjob ${{needs.rust_prepare.result}} "Rust prepare"
+          subjob ${{needs.rust_lint.result}} "Rust lints"
+          subjob ${{needs.rust_check.result}} "Rust checks"
+          subjob ${{needs.turbopack_rust_test1.result}} "Turbopack Rust tests (linux)"


It's weird that this name is different from that of the job itself (Turbopack Rust testing on ubuntu).

We might move more tests from 2 to 1 when they have faster runners... I think it's more part 1 and 2, instead of linux in particular

Maybe rename to (blocking) and (non-blocking)?

tknickman

Love it, thanks!

sokra added 2 commits February 6, 2023 09:06

dry up summary task

83955e4

split tests into two steps, report ok and done separately

dd334b1

github-actions bot added the area: ci Internal CI for vercel/turborepo label Feb 6, 2023

vercel bot deployed to Preview – turbo-site February 6, 2023 08:25 View deployment

sokra added 2 commits February 6, 2023 09:26

fix job names

a2393ce

fixup

9bc1b30

fix bash syntax

4768f14

fix final step

e8f6276

sokra marked this pull request as ready for review February 6, 2023 12:11

sokra requested review from a team as code owners February 6, 2023 12:11

sokra requested review from NicholasLYang, alexkirsz and mehulkar February 6, 2023 12:11

alexkirsz approved these changes Feb 6, 2023

View reviewed changes

tknickman approved these changes Feb 6, 2023

View reviewed changes

sokra merged commit ce48d95 into main Feb 6, 2023

sokra deleted the sokra/ci/split branch February 6, 2023 14:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

split tests into two steps, report ok and done separately #3646

split tests into two steps, report ok and done separately #3646

Uh oh!

sokra commented Feb 6, 2023 •

edited

Loading

Uh oh!

vercel bot commented Feb 6, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Feb 6, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Feb 6, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Feb 6, 2023

Uh oh!

github-actions bot commented Feb 6, 2023

Uh oh!

alexkirsz Feb 6, 2023

Uh oh!

sokra Feb 6, 2023

Uh oh!

alexkirsz Feb 6, 2023

Uh oh!

tknickman left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

split tests into two steps, report ok and done separately #3646

split tests into two steps, report ok and done separately #3646

Uh oh!

Conversation

sokra commented Feb 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Feb 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ CI failed ⚠️

Uh oh!

github-actions bot commented Feb 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Failing next.js integration test suites

Test summary

Uh oh!

github-actions bot commented Feb 6, 2023

Benchmark for e2b894f

Uh oh!

github-actions bot commented Feb 6, 2023

Benchmark for d0c0f4b

Uh oh!

alexkirsz Feb 6, 2023

Choose a reason for hiding this comment

Uh oh!

sokra Feb 6, 2023

Choose a reason for hiding this comment

Uh oh!

alexkirsz Feb 6, 2023

Choose a reason for hiding this comment

Uh oh!

tknickman left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sokra commented Feb 6, 2023 •

edited

Loading

vercel bot commented Feb 6, 2023 •

edited

Loading

github-actions bot commented Feb 6, 2023 •

edited

Loading

github-actions bot commented Feb 6, 2023 •

edited

Loading

Benchmark for `e2b894f`

Benchmark for `d0c0f4b`