[Marvell] Build infrastructure enhancements by pavannaregundi · Pull Request #18143 · sonic-net/sonic-buildimage

pavannaregundi · 2024-02-21T07:23:37Z

Why I did it

This change is about marvell build infrastructure enhancement by merging architecture specific marvell platform(marvell-arm64 and marvell-armhf) folders under 'marvell'.

Work item tracking

Microsoft ADO (number only):

How I did it

This is archived with following changes divided into following commits,

Move mrvl-prestera and sonic-platform-marvell submodule
'mrvl-prestera' and 'sonic-platform-marvell' submodules are moved from 'marvell-arm64'to 'marvell'. Additionaly submodules are changed to support multi-architecture build.
Create sonic-platform-nokia from marvell-arm64/armhf
'sonic-platform-nokia' is created from merge of 'marvell-armhf' and 'marvell-arm64' in 'marvell'. Additionally changes are done in debian/rules to support multi-architecture build.
Change marvell platform makefile to support all arch
Combine all the architecture targets in makefiles. Added CONFIGURED_ARCH check to build for specific architecture.
Updated sai deb which matches latest SAI commit.
Add support for architecture specific platform.conf
Changed build_image.sh and onie-mk-demo.sh to read architecture specific platform.conf.
Moved marvell-arm64/platform.conf to marvell/platform_arm64.conf.
Moved marvell-armhf/platform.conf to marvell/platform_armhf.conf.
[arm64] Adjust the path to read sonic_fit.its
Use CONFIGURED_PLATFORM based path while reading sonic_fit.its.
[arm64/armhf] Add architecure suffix for output image
This change is needed to generate the output image with architecture suffix for arm64/armhf platform.
For arm64, defined a marvell specific file to avoid affect on other vendors.
Change platform_asic name to marvell
Change 'marvell-arm64' and 'marvell-armhf' platform_asic name to 'marvell'.
Remove marvell-armhf and marvell-arm64 folders
Removed 'marvell-armhf' and 'marvell-arm64' as infra is combined with 'marvell' folder.
Update readme to reflect new build commands
Change azure pipeline commands for marvell

New build commands:
AMD64 - make configure PLATFORM=marvell && make target/sonic-marvell.bin
ARM64 - make configure PLATFORM=marvell PLATFORM_ARCH=arm64 && make target/sonic-marvell-arm64.bin
ARMHF - make configure PLATFORM=marvell PLATFORM_ARCH=armhf && make target/sonic-marvell-armhf.bin

How to verify it

Built images for all the platform and installed on hardware to verify the port up.
Verified onie and sonic-sonic install.

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

pavannaregundi · 2024-02-21T10:05:32Z

@Pavan-Nokia please review.

saiarcot895 · 2024-02-26T23:14:04Z

.azure-pipelines/azure-pipelines-image-template.yml

Instead of the if-checks here, can the top-level pipeline build files be updated to point to marvell instead?

@saiarcot895 If we look at the changes in the build commands, only configure command is different in "PLATFORM=marvell" string. Other commands are same. So I opted this change.
If we change name in top-level pipeline( ex: name: marvell-armhf) we need to change or checks multiple places such as how the build pipelines are fetched in "Azure.sonic-buildimage.official.marvell-armhf?branchName=master&label=Marvell-armhf" or while running a make command(make target/sonic-marvell-armhf.bin) for build. I thought this is less change, and does not break other arm64 platform builds.

hmm, @xumia, thoughts?

How about define a new parameter in azure-pipelines-build.yml for marvell-armhf and marvell-amd64.
https://github.com/sonic-net/sonic-buildimage/blob/7f454a778a3f324304c241ffcbb4fc012bf75c90/.azure-pipelines/azure-pipelines-build.yml#L100
For example:

PLATFORM: marvell

If the variable PLATFORM is not defined (empty), then use PLATFORM_AZP

[ -z "$PLATFORM" ] && PLATFORM=$(PLATFORM_AZP) ENABLE_DOCKER_BASE_PULL=y make PLATFORM=$PLATFORM PLATFORM_ARCH=$(PLATFORM_ARCH) $(BUILD_OPTIONS) configure // There is a little different $PLATFORM and $(PLATFORM), be careful.

Any other platforms have the similar requirements, only need to override the variable, no logic change.

@xumia Have pushed the changes as recommended by you. Please review.

xumia · 2024-03-01T04:58:23Z

.azure-pipelines/build-template.yml

The file has been deprecated, and it is no use. The change can be skipped.

Ok. Reverting this change.

pavannaregundi · 2024-03-05T03:43:48Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2024-03-05T03:43:51Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2024-03-05T03:43:57Z

Pull request contains merge conflicts.

Blueve · 2024-03-06T12:54:54Z

@liushilongbuaa

Blueve · 2024-03-07T06:16:34Z

Hi @pavannaregundi , can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

pavannaregundi · 2024-03-07T06:18:25Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2024-03-07T06:18:27Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2024-03-07T06:18:38Z

Azure Pipelines successfully started running 1 pipeline(s).

liushilongbuaa · 2024-03-18T09:26:05Z

.azure-pipelines/azure-pipelines-image-template.yml

          sudo apt-get install -y acl
          sudo bash -c "echo 1 > /proc/sys/vm/compact_memory"
-          ENABLE_DOCKER_BASE_PULL=y make PLATFORM=$(PLATFORM_AZP) PLATFORM_ARCH=$(PLATFORM_ARCH) $(BUILD_OPTIONS) configure
+          [ -z "$PLATFORM" ] && PLATFORM=$(PLATFORM_AZP)


Why? When PLATFORM is set?

@liushilongbuaa This is a new optional variable introduced in 'azure-pipelines-build.yml' to select the platform if it is different from default PLATFORM_AZP. Basically, pick platform name from PLATFORM_AZP when PLATFORM variable is not set.

Details of this change are already present in comments above.

liushilongbuaa · 2024-03-18T09:28:48Z

build_debian.sh

            sudo LANG=C chroot $FILESYSTEM_ROOT mv /boot/u${INITRD_FILE} /boot/$INITRD_FILE
        else
-            sudo cp -v $PLATFORM_DIR/${sonic_asic_platform}-${CONFIGURED_ARCH}/sonic_fit.its $FILESYSTEM_ROOT/boot/
+            sudo cp -v $PLATFORM_DIR/$CONFIGURED_PLATFORM/sonic_fit.its $FILESYSTEM_ROOT/boot/


centec-arm64 is also impacted.

Yes. But $CONFIGURED_PLATFORM for "centec-arm64" is "centec-arm64". So change will work.

liushilongbuaa · 2024-03-18T10:32:22Z

can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

pavannaregundi · 2024-03-18T10:38:57Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2024-03-18T10:38:59Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2024-03-18T10:39:12Z

Azure Pipelines successfully started running 1 pipeline(s).

liushilongbuaa · 2024-03-19T09:58:27Z

can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

Done. Build still failing.

Can you debug locally to fix sonic-config-engine UT?

liushilongbuaa · 2024-03-20T09:34:19Z

Hi @neethajohn , can you help on the UT?

pavannaregundi · 2024-03-20T11:57:41Z

can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

Done. Build still failing.

Can you debug locally to fix sonic-config-engine UT?

@liushilongbuaa We will try looking at this. However, someone who has experience in this area can help us will be good.

radha-danda · 2024-04-03T02:07:49Z

/azpw run Azure.sonic-buildimage

Blueve · 2024-04-08T09:30:45Z

Potential fix of build ut issue: #18566

The marvell build pipeline continuously fail, seems related to the new added PLATFORM variable. And above PR can clean up the variable before build py-wheel

liushilongbuaa · 2024-04-08T10:20:40Z

@pavannaregundi , can you rerun PR checker by comment '/azpw run Azure.sonic-buildimage'?

'mrvl-prestera' and 'sonic-platform-marvell' submodules are moved from 'marvell-arm64'to 'marvell'. Additionaly submodules are changed to support multi-architecture build. Signed-off-by: Pavan Naregundi <[email protected]>

sonic-platform-nokia is created from merge of 'marvell-armhf' and 'marvell-arm64' in 'marvell'. Additionaly changes are done to support multi-arch build. Signed-off-by: Pavan Naregundi <[email protected]>

Combine all the architecture targets in makefiles. Added CONFIGURED_ARCH check to build for specific architecture. Updated sai deb which matches latest SAI commit. Signed-off-by: Pavan Naregundi <[email protected]>

Changed build_image.sh and onie-mk-demo.sh to read architecture specific platform.conf. Moved marvell-arm64/platform.conf to marvell/platform_arm64.conf. Moved marvell-armhf/platform.conf to marvell/platform_armhf.conf. Signed-off-by: Pavan Naregundi <[email protected]>

Use CONFIGURED_PLATFORM based path while reading sonic_fit.its. Signed-off-by: Pavan Naregundi <[email protected]>

This change is needed to generate the output image with architecture suffix for arm64/armhf platform. For arm64, defined a marvell specific file to avoid affect on other vendors. Signed-off-by: Pavan Naregundi <[email protected]>

Change 'marvell-arm64' and 'marvell-armhf' platform_asic name to 'marvell'. Signed-off-by: Pavan Naregundi <[email protected]>

Removed 'marvell-armhf' and 'marvell-arm64' as infra is combined with 'marvell' folder. Signed-off-by: Pavan Naregundi <[email protected]>

Signed-off-by: Pavan Naregundi <[email protected]>

make configure is changed as below for 'marvell-arm64' and 'marvell-armhf'. arm64 - make configure PLATFORM=marvell PLATFORM_ARCH=arm64 armhf - make configure PLATFORM=marvell PLATFORM_ARCH=armhf Signed-off-by: Pavan Naregundi <[email protected]>

pavannaregundi · 2024-04-09T05:02:59Z

@Blueve @liushilongbuaa I have pushed a change by changing the azure new variable for platform to "PLATFORM_NAME", instead of "PLATFORM". It could be that yml is setting this as env variable.

pavannaregundi · 2024-04-09T10:48:00Z

@Blueve @liushilongbuaa I have pushed a change by changing the azure new variable for platform to "PLATFORM_NAME", instead of "PLATFORM". It could be that yml is setting this as env variable.

Builds are passing now.

liushilongbuaa

LGTM

pavannaregundi · 2024-04-12T07:02:58Z

@prgeor All comments are addressed. Can we merge this PR.

prgeor · 2024-04-14T07:47:29Z

@prgeor All comments are addressed. Can we merge this PR.

@lguohan @yxieca can you merge?

pavannaregundi requested a review from lguohan as a code owner February 21, 2024 07:23

pavannaregundi force-pushed the master branch 3 times, most recently from 90b6ff5 to 2174f65 Compare February 22, 2024 02:50

saiarcot895 reviewed Feb 26, 2024

View reviewed changes

saiarcot895 requested review from prgeor and xumia February 28, 2024 04:11

pavannaregundi force-pushed the master branch from 2174f65 to 7f454a7 Compare February 29, 2024 08:45

xumia reviewed Mar 1, 2024

View reviewed changes

pavannaregundi force-pushed the master branch 3 times, most recently from 6d0f3b5 to 56d0df8 Compare March 4, 2024 03:38

pavannaregundi force-pushed the master branch from 56d0df8 to 5a0b93e Compare March 5, 2024 04:46

Blueve requested a review from liushilongbuaa March 6, 2024 12:55

pavannaregundi force-pushed the master branch from 5a0b93e to cb9c724 Compare March 13, 2024 03:44

liushilongbuaa reviewed Mar 18, 2024

View reviewed changes

krismarvell mentioned this pull request Apr 8, 2024

Fix for sonic-config-engine build failure - test_buffers_dell6100_render_template #18566

Open

9 tasks

pavannaregundi added 9 commits April 9, 2024 04:27

Move mrvl-prestera and sonic-platform-marvell submodule

531a0e6

'mrvl-prestera' and 'sonic-platform-marvell' submodules are moved from 'marvell-arm64'to 'marvell'. Additionaly submodules are changed to support multi-architecture build. Signed-off-by: Pavan Naregundi <[email protected]>

Create sonic-platform-nokia from marvell-arm64/armhf

8bfd373

sonic-platform-nokia is created from merge of 'marvell-armhf' and 'marvell-arm64' in 'marvell'. Additionaly changes are done to support multi-arch build. Signed-off-by: Pavan Naregundi <[email protected]>

Change marvell platform makefile to support all arch

355c8a2

Combine all the architecture targets in makefiles. Added CONFIGURED_ARCH check to build for specific architecture. Updated sai deb which matches latest SAI commit. Signed-off-by: Pavan Naregundi <[email protected]>

[arm64] Adjust the path to read sonic_fit.its

d663b53

Use CONFIGURED_PLATFORM based path while reading sonic_fit.its. Signed-off-by: Pavan Naregundi <[email protected]>

[arm64/armhf] Add architecture suffix for output image

a295051

This change is needed to generate the output image with architecture suffix for arm64/armhf platform. For arm64, defined a marvell specific file to avoid affect on other vendors. Signed-off-by: Pavan Naregundi <[email protected]>

Change platform_asic name to marvell

295a7ab

Change 'marvell-arm64' and 'marvell-armhf' platform_asic name to 'marvell'. Signed-off-by: Pavan Naregundi <[email protected]>

Remove marvell-armhf and marvell-arm64 folders

a0cc78d

Removed 'marvell-armhf' and 'marvell-arm64' as infra is combined with 'marvell' folder. Signed-off-by: Pavan Naregundi <[email protected]>

Update readme to reflect new build commands

e9423ab

Signed-off-by: Pavan Naregundi <[email protected]>

pavannaregundi force-pushed the master branch 3 times, most recently from 793f219 to 44e9f1a Compare April 9, 2024 04:58

pavannaregundi force-pushed the master branch from 44e9f1a to 4611de1 Compare April 9, 2024 05:00

Blueve approved these changes Apr 9, 2024

View reviewed changes

saiarcot895 requested a review from liushilongbuaa April 10, 2024 01:28

liushilongbuaa approved these changes Apr 10, 2024

View reviewed changes

prgeor approved these changes Apr 14, 2024

View reviewed changes

yxieca merged commit 006f4b2 into sonic-net:master Apr 15, 2024

Conversation

pavannaregundi commented Feb 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

Work item tracking

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Uh oh!

pavannaregundi commented Feb 21, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xumia Mar 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pavannaregundi commented Mar 5, 2024

Uh oh!

mssonicbld commented Mar 5, 2024

Uh oh!

azure-pipelines bot commented Mar 5, 2024

Uh oh!

Blueve commented Mar 6, 2024

Uh oh!

Blueve commented Mar 7, 2024

Uh oh!

pavannaregundi commented Mar 7, 2024

Uh oh!

mssonicbld commented Mar 7, 2024

Uh oh!

azure-pipelines bot commented Mar 7, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liushilongbuaa commented Mar 18, 2024

Uh oh!

pavannaregundi commented Mar 18, 2024

Uh oh!

mssonicbld commented Mar 18, 2024

Uh oh!

azure-pipelines bot commented Mar 18, 2024

Uh oh!

liushilongbuaa commented Mar 19, 2024

Uh oh!

liushilongbuaa commented Mar 20, 2024

Uh oh!

pavannaregundi commented Mar 20, 2024

Uh oh!

radha-danda commented Apr 3, 2024

Uh oh!

Blueve commented Apr 8, 2024

Uh oh!

liushilongbuaa commented Apr 8, 2024

Uh oh!

pavannaregundi commented Apr 9, 2024

Uh oh!

pavannaregundi commented Apr 9, 2024

Uh oh!

liushilongbuaa left a comment

pavannaregundi commented Feb 21, 2024 •

edited

Loading

xumia Mar 1, 2024 •

edited

Loading