Refactor chakra_converter and Add Tests

[TaekyungHeo]

Summary

• Refactor code
• Add tests using pytest
• Set up GitHub Actions for pytest

Test Plan

1. Pytests

$ python -m pytest -vv tests                           
=========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.10.2, pytest-8.1.1, pluggy-1.5.0 -- /Users/theo/venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/theo/chakra
configfile: pyproject.toml
plugins: typeguard-2.13.3, mock-3.14.0, anyio-4.2.0
collected 6 items                                                                                                                                                         

tests/converter/test_pytorch_node.py::test_pytorch_node_parsing PASSED                                                                                              [ 16%]
tests/converter/test_pytorch_tensor.py::test_pytorch_tensor_initialization PASSED                                                                                   [ 33%]
tests/converter/test_pytorch_tensor.py::test_pytorch_tensor_is_valid PASSED                                                                                         [ 50%]
tests/converter/test_pytorch_tensor.py::test_pytorch_tensor_properties PASSED                                                                                       [ 66%]
tests/converter/test_pytorch_tensor.py::test_pytorch_tensor_has_valid_storage_id PASSED                                                                             [ 83%]
tests/converter/test_pytorch_tensor.py::test_list_to_pytorch_tensor PASSED                                                                                          [100%]

============================================================================ 6 passed in 0.07s ============================================================================

2. Correlation
2.1 Run trace_link

chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_0.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_0.json --output-file ~/megatron_0.json &
chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_1.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_1.json --output-file ~/megatron_1.json &
chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_2.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_2.json --output-file ~/megatron_2.json &
chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_3.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_3.json --output-file ~/megatron_3.json &
chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_4.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_4.json --output-file ~/megatron_4.json &
chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_5.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_5.json --output-file ~/megatron_5.json &
chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_6.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_6.json --output-file ~/megatron_6.json &
chakra_trace_link --pytorch-et-file /Users/theo/Downloads/llama_pytorch24.05/megatron_et_7.json --kineto-file /Users/theo/Downloads/llama_pytorch24.05/megatron_kineto_7.json --output-file ~/megatron_7.json &

2.2 Run et_converter

chakra_converter --input_filename ~/megatron_0.json --output_filename megatron_0.chakra --input_type PyTorch > /tmp/rank_0 &
chakra_converter --input_filename ~/megatron_1.json --output_filename megatron_1.chakra --input_type PyTorch > /tmp/rank_1 &
chakra_converter --input_filename ~/megatron_2.json --output_filename megatron_2.chakra --input_type PyTorch > /tmp/rank_2 &
chakra_converter --input_filename ~/megatron_3.json --output_filename megatron_3.chakra --input_type PyTorch > /tmp/rank_3 &
chakra_converter --input_filename ~/megatron_4.json --output_filename megatron_4.chakra --input_type PyTorch > /tmp/rank_4 &
chakra_converter --input_filename ~/megatron_5.json --output_filename megatron_5.chakra --input_type PyTorch > /tmp/rank_5 &
chakra_converter --input_filename ~/megatron_6.json --output_filename megatron_6.chakra --input_type PyTorch > /tmp/rank_6 &
chakra_converter --input_filename ~/megatron_7.json --output_filename megatron_7.chakra --input_type PyTorch > /tmp/rank_7 &

2.3 Results

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[TaekyungHeo]

Converting to draft because I am unsure whether it is acceptable to add a trace as is without using Git LFS.

[srinivas212]

lgtm. discussed offline.