Skip to content

Add --trace-file option to output catapult traces #1639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Dec 13, 2018
Merged

Add --trace-file option to output catapult traces #1639

merged 14 commits into from
Dec 13, 2018

Conversation

@emillon
Copy link
Collaborator

@emillon emillon commented Dec 11, 2018

These traces can be loaded in different reporting tools such as chrome://tracing. At the moment this reports info about subprocesses and some GC stats, but this is generic enough to report info about phases, etc.

See #1180.

As for the PR itself, I'd like to make the closing explicit, but we don't have a central "at exit" place in main.ml I think. I'd also like to add some tests for this before merging but I'd be happy to have feedback even before that.

@emillon emillon requested a review from rgrinberg December 11, 2018 11:28
rgrinberg
rgrinberg reviewed Dec 11, 2018
View reviewed changes
rgrinberg
rgrinberg reviewed Dec 11, 2018
View reviewed changes
rgrinberg
rgrinberg reviewed Dec 11, 2018
View reviewed changes
rgrinberg
rgrinberg reviewed Dec 11, 2018
View reviewed changes
@emillon emillon requested a review from a user December 11, 2018 15:30
@rgrinberg
Copy link
Member

rgrinberg commented Dec 11, 2018

Unless we are versioning catapult, I think we should also mark is as "experimental" like we do for the other libs for now.

@emillon
Copy link
Collaborator Author

emillon commented Dec 11, 2018

Catapult does not have a public_name though, so my understanding is that it can not be used outside of dune (like xdg). Is this enough?

@rgrinberg
Copy link
Member

rgrinberg commented Dec 11, 2018

Ah ok. Then yeah it should be fine.

ghost
ghost suggested changes Dec 11, 2018
View reviewed changes
@emillon emillon changed the title WIP: Add --trace-file option to output catapult traces Add --trace-file option to output catapult traces Dec 12, 2018
@emillon
Copy link
Collaborator Author

emillon commented Dec 12, 2018

I added docs and unit tests. The last thing I'm not too sure about is the --trace-file name. Any ideas to make this better?

@rgrinberg
Copy link
Member

rgrinberg commented Dec 12, 2018

In shake this is called --profile=report.trace [1]. We can't really use profile as we already have something that enabled this. So I think a --trace-file option is fine.

https://shakebuild.com/profiling#chrome-traces

emillon added 12 commits December 13, 2018 21:35
@emillon
Add --trace-file option to output catapult traces
248a3e9 
These traces can be loaded in different reporting tools such as
`chrome://tracing`. At the moment this reports info about subprocesses
and some GC stats, but this is generic enough to report info about
phases, etc.

Closes #1180

Signed-off-by: Etienne Millon <[email protected]>
@emillon
Split catapult into its own lib
eda9764 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Only work on channels
a0fb5c3 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Add some tests
a6d1225 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Document the interface
c5bcda1 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Do not append to trace files
aa5b18a 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Document the feature
f85f391 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Do not call gettimeofday when disabled
4f165a0 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Print with explicit functions
be81143 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Add a unit test for catapult
0357ca0 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Use the same colors for non-opt commands
9dd75a0 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Use with_process
84ac45d 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Simplify the state machine
ae242c0 
Signed-off-by: Etienne Millon <[email protected]>
@emillon
Add changelog entry
587d413 
Signed-off-by: Etienne Millon <[email protected]>
@emillon emillon merged commit 0ca7d42 into master Dec 13, 2018
@emillon emillon deleted the catapult branch December 13, 2018 20:36
rgrinberg added a commit to rgrinberg/opam-repository that referenced this pull request Feb 12, 2019
@rgrinberg
[new release] dune (1.7.0)
674b1ed 
CHANGES:

- Second step of the deprecation of jbuilder: the `jbuilder` binary
  now emits a warning on every startup and both `jbuilder` and `dune`
  emit warnings when encountering `jbuild` files (ocaml/dune#1752, @diml)

- Change the layout of build artifacts inside _build. The new layout enables
  optimizations that depend on the presence of `.cmx` files of private modules
  (ocaml/dune#1676, @bobot)

- Fix merlin handling of private module visibility (ocaml/dune#1653 @bobot)

- unstable-fmt: use boxes to wrap some lists (ocaml/dune#1608, fix ocaml/dune#1153, @emillon,
  thanks to @rgrinberg)

- skip directories when looking up programs in the PATH (ocaml/dune#1628, fixes
  ocaml/dune#1616, @diml)

- Use `lsof` on macOS to implement `--stats` (ocaml/dune#1636, fixes ocaml/dune#1634, @xclerc)

- Generate `dune-package` files for every package. These files are installed and
  read instead of `META` files whenever they are available (ocaml/dune#1329, @rgrinberg)

- Fix preprocessing for libraries with `(include_subdirs ..)` (ocaml/dune#1624, fix ocaml/dune#1626,
  @nojb, @rgrinberg)

- Do not generate targets for archive that don't match the `modes` field.
  (ocaml/dune#1632, fix ocaml/dune#1617, @rgrinberg)

- When executing actions, open files lazily and close them as soon as
  possible in order to reduce the maximum number of file descriptors
  opened by Dune (ocaml/dune#1635, ocaml/dune#1643, fixes ocaml/dune#1633, @jonludlam, @rgrinberg,
  @diml)

- Reimplement the core of Dune using a new generic memoization system
  (ocaml/dune#1489, @rudihorn, @diml)

- Replace the broken cycle detection algorithm by a state of the art
  one from [this paper](https://doi.org/10.1145/2756553) (ocaml/dune#1489,
  @rudihorn)

- Get the correct environment node for multi project workspaces (ocaml/dune#1648,
  @rgrinberg)

- Add `dune compute` to call internal memoized functions (ocaml/dune#1528,
  @rudihorn, @diml)

- Add `--trace-file` option to trace dune internals (ocaml/dune#1639, fix ocaml/dune#1180, @emillon)

- Add `--no-print-directory` (borrowed from GNU make) to suppress
  `Entering directory` messages. (ocaml/dune#1668, @dra27)

- Remove `--stats` and track fd usage in `--trace-file` (ocaml/dune#1667, @emillon)

- Add virtual libraries feature and enable it by default (ocaml/dune#1430 fixes ocaml/dune#921,
  @rgrinberg)

- Fix handling of Control+C in watch mode (ocaml/dune#1678, fixes ocaml/dune#1671, @diml)

- Look for jsoo runtime in the same dir as the `js_of_ocaml` binary
  when the ocamlfind package is not available (ocaml/dune#1467, @nojb)

- Make the `seq` package available for OCaml >= 4.07 (ocaml/dune#1714, @rgrinberg)

- Add locations to error messages where a rule fails to generate targets and
  rules that require files outside the build/source directory. (ocaml/dune#1708, fixes
  ocaml/dune#848, @rgrinberg)

- Let `Configurator` handle `sizeof` (in addition to negative numbers).
  (ocaml/dune#1726, fixes ocaml/dune#1723, @Chris00)

- Fix an issue causing menhir generated parsers to fail to build in
  some cases. The fix is to systematically use `-short-paths` when
  calling `ocamlc -i` (ocaml/dune#1743, fix ocaml/dune#1504, @diml)

- Never raise when printing located errors. The code that would print the
  location excerpts was prone to raising. (ocaml/dune#1744, fix ocaml/dune#1736, @rgrinberg)

- Add a `dune upgrade` command for upgrading jbuilder projects to Dune
  (ocaml/dune#1749, @diml)

- When automatically creating a `dune-project` file, insert the
  detected name in it (ocaml/dune#1749, @diml)

- Add `(implicit_transitive_deps <bool>)` mode to dune projects. When this mode
  is turned off, transitive dependencies are not accessible. Only listed
  dependencies are directly accessible. (ocaml/dune#1734, ocaml/dune#430, @rgrinberg, @hnrgrgr)

- Add `toplevel` stanza. This stanza is used to define toplevels with libraries
  already preloaded. (ocaml/dune#1713, @rgrinberg)

- Generate `.merlin` files that account for normal preprocessors defined using a
  subset of the `action` language. (ocaml/dune#1768, @rgrinberg)

- Emit `(orig_src_dir <path>)` metadata in `dune-package` for dune packages
  built with `--store-orig-source-dir` command line flag (also controlled by
  `DUNE_STORE_ORIG_SOURCE_DIR` env variable). This is later used to generate
  `.merlin` with `S`-directives pointed to original source locations and thus
  allowing merlin to see those. (ocaml/dune#1750, @andreypopp)

- Improve the behavior of `dune promote` when the files to be promoted have been
  deleted. (ocaml/dune#1775, fixes ocaml/dune#1772, @diml)

- unstable-fmt: preserve comments (ocaml/dune#1766, @emillon)

- Pass flags correctly when using `staged_pps` (ocaml/dune#1779, fixes ocaml/dune#1774, @diml)

- Fix an issue with the use of `(mode promote)` in the menhir
  stanza. It was previously causing intermediate *mock* files to be
  promoted (ocaml/dune#1783, fixes ocaml/dune#1781, @diml)

- unstable-fmt: ignore files using OCaml syntax (ocaml/dune#1784, @emillon)

- Configurator: Add `which` function to replace the `which` command line utility
  in a cross platform way. (ocaml/dune#1773, fixes ocaml/dune#1705, @Chris00)

- Make configurator append paths to `$PKG_CONFIG_PATH` on macOS. Previously it
  was prepending paths and thus `$PKG_CONFIG_PATH` set by users could have been
  overridden by homebrew installed libraries (ocaml/dune#1785, @andreypopp)

- Disallow c/cxx sources that share an object file in the same stubs archive.
  This means that `foo.c` and `foo.cpp` can no longer exist in the same library.
  (ocaml/dune#1788, @rgrinberg)

- Forbid use of `%{targets}` (or `${@}` in jbuild files) inside
  preprocessing actions
  (ocaml/dune#1812, fixes ocaml/dune#1811, @diml)

- Add `DUNE_PROFILE` environment variable to easily set the profile. (ocaml/dune#1806,
  @rgrinberg)

- Deprecate the undocumented `(no_keep_locs)` field. It was only
  necessary until virtual libraries were supported (ocaml/dune#1822, fix ocaml/dune#1816,
  @diml)

- Rename `unstable-fmt` to `format-dune-file` and remove its `--inplace` option.
  (ocaml/dune#1821, @emillon).

- Autoformatting: `(using fmt 1.1)` will also format dune files (ocaml/dune#1821, @emillon).

- Autoformatting: record dependencies on `.ocamlformat-ignore` files (ocaml/dune#1824,
  fixes ocaml/dune#1793, @emillon)
@rgrinberg rgrinberg mentioned this pull request Feb 12, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rgrinberg rgrinberg rgrinberg approved these changes

+1 more reviewer
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@emillon @rgrinberg