Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support clang build coverage execution for unit tests #37563

Merged
merged 8 commits into from
Feb 14, 2025
Merged

Support clang build coverage execution for unit tests #37563

merged 8 commits into from
Feb 14, 2025
+185 −2

Conversation

andy31415
Copy link
Contributor

Changes

  • Added logic for unit tests to define the clang profraw output to depend on the test binary name for unit tests (so we have separate profraw files instead of "default.profraw")
  • Added support for generating html coverage report in build_examples
  • slight change to build_coverage to make it work on more machines.

Testing

Built linux-x64-tests-clang-coverage and observed html output.
Compared HTML output with the build_coverage.sh and it seems reasonably similar, except build_coverage seems to miss some files (e.g. no clusters report even though they have unit tests) and clang seems to consider more lines (e.g. curly braces).

image

@semanticdiff-com SemanticDiff.com
Copy link

semanticdiff-com bot commented Feb 13, 2025 •
edited
Loading

Review changes with  SemanticDiff

Changed Files
File Status
  scripts/build/build/targets.py  62% smaller
  build/chip/chip_test_suite.gni Unsupported file format
  build/config/compiler/BUILD.gn Unsupported file format
  scripts/build/builders/host.py  0% smaller
  scripts/build/clang_coverage_wrapper.py  0% smaller
  scripts/build_coverage.sh Unsupported file format

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 13, 2025 •
edited
Loading

PR #37563: Size comparison from ecb3c14 to 16acb79

Full report (72 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section ecb3c14 16acb79 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1095960 1095960 0 0.0
RAM 94906 94906 0 0.0
bl702 lighting-app bl702+eth FLASH 652080 652080 0 0.0
RAM 33633 33633 0 0.0
bl702+wifi FLASH 828572 828572 0 0.0
RAM 22341 22341 0 0.0
bl706+mfd+rpc+littlefs FLASH 1061718 1061718 0 0.0
RAM 32285 32285 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 892592 892592 0 0.0
RAM 26912 26912 0 0.0
lighting-app bl702l+mfd+littlefs FLASH 975962 975962 0 0.0
RAM 24752 24752 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 815204 815204 0 0.0
RAM 120352 120352 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 823832 823832 0 0.0
RAM 125360 125360 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 770992 770992 0 0.0
RAM 113820 113820 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 755260 755260 0 0.0
RAM 114028 114028 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 538894 538894 0 0.0
RAM 205208 205208 0 0.0
lock CC3235SF_LAUNCHXL FLASH 572766 572766 0 0.0
RAM 205360 205360 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 656501 656501 0 0.0
RAM 75420 75420 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 676361 676361 0 0.0
RAM 78060 78060 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 676361 676361 0 0.0
RAM 78060 78060 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 633285 633285 0 0.0
RAM 70488 70488 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 616117 616117 0 0.0
RAM 71532 71532 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 635753 635753 0 0.0
RAM 74076 74076 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 635753 635753 0 0.0
RAM 74076 74076 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 635621 635621 0 0.0
RAM 74540 74540 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 655337 655337 0 0.0
RAM 77084 77084 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 655337 655337 0 0.0
RAM 77084 77084 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 611969 611969 0 0.0
RAM 68628 68628 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 631829 631829 0 0.0
RAM 71268 71268 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 631829 631829 0 0.0
RAM 71268 71268 0 0.0
efr32 lock-app BRD4187C FLASH 937264 937264 0 0.0
RAM 159908 159908 0 0.0
BRD4338a FLASH 730592 730584 -8 -0.0
RAM 234720 234720 0 0.0
window-app BRD4187C FLASH 1029856 1029856 0 0.0
RAM 128012 128012 0 0.0
esp32 all-clusters-app c3devkit DRAM 97312 97312 0 0.0
FLASH 1581680 1581680 0 0.0
IRAM 83820 83820 0 0.0
m5stack DRAM 116100 116100 0 0.0
FLASH 1549606 1549606 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4760 4760 0 0.0
FLASH 2708747 2708747 0 0.0
RAM 132784 132784 0 0.0
all-clusters-app debug unknown 5568 5568 0 0.0
FLASH 5975210 5975210 0 0.0
RAM 531600 531600 0 0.0
all-clusters-minimal-app debug unknown 5464 5464 0 0.0
FLASH 5322754 5322754 0 0.0
RAM 242712 242712 0 0.0
bridge-app debug unknown 5480 5480 0 0.0
FLASH 4681398 4681398 0 0.0
RAM 221448 221448 0 0.0
chip-tool debug unknown 6120 6120 0 0.0
FLASH 13098960 13098960 0 0.0
RAM 596578 596578 0 0.0
chip-tool-ipv6only arm64 unknown 21816 21816 0 0.0
FLASH 11162224 11162224 0 0.0
RAM 648256 648256 0 0.0
fabric-admin debug unknown 5808 5808 0 0.0
FLASH 11387977 11387977 0 0.0
RAM 596362 596362 0 0.0
fabric-bridge-app debug unknown 4736 4736 0 0.0
FLASH 4506720 4506720 0 0.0
RAM 208632 208632 0 0.0
fabric-sync debug unknown 4976 4976 0 0.0
FLASH 5612965 5612965 0 0.0
RAM 483504 483504 0 0.0
lighting-app debug+rpc+ui unknown 6152 6152 0 0.0
FLASH 5484513 5484513 0 0.0
RAM 225392 225392 0 0.0
lock-app debug unknown 5416 5416 0 0.0
FLASH 4730794 4730794 0 0.0
RAM 207696 207696 0 0.0
ota-provider-app debug unknown 4776 4776 0 0.0
FLASH 4359828 4359828 0 0.0
RAM 201336 201336 0 0.0
ota-requestor-app debug unknown 4728 4728 0 0.0
FLASH 4497204 4497204 0 0.0
RAM 205920 205920 0 0.0
shell debug unknown 4256 4256 0 0.0
FLASH 3005468 3005468 0 0.0
RAM 160552 160552 0 0.0
thermostat-no-ble arm64 unknown 9512 9512 0 0.0
FLASH 4096488 4096488 0 0.0
RAM 246024 246024 0 0.0
tv-app debug unknown 5744 5744 0 0.0
FLASH 5951893 5951893 0 0.0
RAM 606904 606904 0 0.0
tv-casting-app debug unknown 5320 5320 0 0.0
FLASH 11271725 11271725 0 0.0
RAM 710864 710864 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 907464 907464 0 0.0
RAM 142411 142411 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 901920 901920 0 0.0
RAM 124755 124755 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 846084 846084 0 0.0
RAM 141339 141339 0 0.0
nxp contact k32w0+release FLASH 584752 584752 0 0.0
RAM 70876 70876 0 0.0
mcxw71+release FLASH 600208 600208 0 0.0
RAM 63096 63096 0 0.0
light k32w0+release FLASH 611076 611076 0 0.0
RAM 70268 70268 0 0.0
k32w1+release FLASH 685512 685512 0 0.0
RAM 48680 48680 0 0.0
lock mcxw71+release FLASH 749024 749024 0 0.0
RAM 67500 67500 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1646756 1646756 0 0.0
RAM 211576 211576 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1553524 1553524 0 0.0
RAM 208392 208392 0 0.0
light cy8ckit_062s2_43012 FLASH 1439196 1439196 0 0.0
RAM 197144 197144 0 0.0
lock cy8ckit_062s2_43012 FLASH 1467412 1467412 0 0.0
RAM 224704 224704 0 0.0
qpg lighting-app qpg6105+debug FLASH 662340 662340 0 0.0
RAM 105220 105220 0 0.0
lock-app qpg6105+debug FLASH 620136 620136 0 0.0
RAM 99664 99664 0 0.0
stm32 light STM32WB5MM-DK FLASH 459736 459736 0 0.0
RAM 141568 141568 0 0.0
telink bridge-app tl7218x FLASH 665226 665226 0 0.0
RAM 90828 90828 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 621908 621908 0 0.0
RAM 31488 31488 0 0.0
light-app-ota-shell-factory-data tl3218x FLASH 745418 745418 0 0.0
RAM 40496 40496 0 0.0
tl7218x FLASH 753974 753974 0 0.0
RAM 97632 97632 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 680678 680678 0 0.0
RAM 52192 52192 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 709240 709240 0 0.0
RAM 73400 73400 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 600796 600796 0 0.0
RAM 138912 138912 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 788928 788928 0 0.0
RAM 96488 96488 0 0.0
tizen all-clusters-app arm unknown 5104 5104 0 0.0
FLASH 1751708 1751708 0 0.0
RAM 93508 93508 0 0.0
chip-tool-ubsan arm unknown 11396 11396 0 0.0
FLASH 18683110 18683110 0 0.0
RAM 8181084 8181084 0 0.0

@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 13, 2025 •
edited
Loading

PR #37563: Size comparison from 96bf5df to 91b752d

Full report (72 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 96bf5df 91b752d change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1095960 1095960 0 0.0
RAM 94906 94906 0 0.0
bl702 lighting-app bl702+eth FLASH 652080 652080 0 0.0
RAM 33633 33633 0 0.0
bl702+wifi FLASH 828572 828572 0 0.0
RAM 22341 22341 0 0.0
bl706+mfd+rpc+littlefs FLASH 1061718 1061718 0 0.0
RAM 32285 32285 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 892592 892592 0 0.0
RAM 26912 26912 0 0.0
lighting-app bl702l+mfd+littlefs FLASH 975962 975962 0 0.0
RAM 24752 24752 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 815204 815204 0 0.0
RAM 120352 120352 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 823832 823832 0 0.0
RAM 125360 125360 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 770992 770992 0 0.0
RAM 113820 113820 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 755260 755260 0 0.0
RAM 114028 114028 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 538894 538894 0 0.0
RAM 205208 205208 0 0.0
lock CC3235SF_LAUNCHXL FLASH 572766 572766 0 0.0
RAM 205360 205360 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 656501 656501 0 0.0
RAM 75420 75420 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 676361 676361 0 0.0
RAM 78060 78060 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 676361 676361 0 0.0
RAM 78060 78060 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 633285 633285 0 0.0
RAM 70488 70488 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 616117 616117 0 0.0
RAM 71532 71532 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 635753 635753 0 0.0
RAM 74076 74076 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 635753 635753 0 0.0
RAM 74076 74076 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 635621 635621 0 0.0
RAM 74540 74540 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 655337 655337 0 0.0
RAM 77084 77084 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 655337 655337 0 0.0
RAM 77084 77084 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 611969 611969 0 0.0
RAM 68628 68628 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 631829 631829 0 0.0
RAM 71268 71268 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 631829 631829 0 0.0
RAM 71268 71268 0 0.0
efr32 lock-app BRD4187C FLASH 937264 937264 0 0.0
RAM 159908 159908 0 0.0
BRD4338a FLASH 730592 730584 -8 -0.0
RAM 234720 234720 0 0.0
window-app BRD4187C FLASH 1029856 1029856 0 0.0
RAM 128012 128012 0 0.0
esp32 all-clusters-app c3devkit DRAM 97312 97312 0 0.0
FLASH 1581680 1581680 0 0.0
IRAM 83820 83820 0 0.0
m5stack DRAM 116100 116100 0 0.0
FLASH 1549606 1549606 0 0.0
IRAM 117039 117039 0 0.0
linux air-purifier-app debug unknown 4712 4712 0 0.0
FLASH 2648901 2648901 0 0.0
RAM 111600 111600 0 0.0
all-clusters-app debug unknown 5520 5520 0 0.0
FLASH 5915332 5915332 0 0.0
RAM 510416 510416 0 0.0
all-clusters-minimal-app debug unknown 5416 5416 0 0.0
FLASH 5262876 5262876 0 0.0
RAM 221528 221528 0 0.0
bridge-app debug unknown 5432 5432 0 0.0
FLASH 4621552 4621552 0 0.0
RAM 200264 200264 0 0.0
chip-tool debug unknown 6088 6088 0 0.0
FLASH 13070970 13070970 0 0.0
RAM 596482 596482 0 0.0
chip-tool-ipv6only arm64 unknown 21816 21816 0 0.0
FLASH 11137744 11137744 0 0.0
RAM 648160 648160 0 0.0
fabric-admin debug unknown 5776 5776 0 0.0
FLASH 11359987 11359987 0 0.0
RAM 596266 596266 0 0.0
fabric-bridge-app debug unknown 4680 4680 0 0.0
FLASH 4446874 4446874 0 0.0
RAM 187480 187480 0 0.0
fabric-sync debug unknown 4936 4936 0 0.0
FLASH 5563813 5563813 0 0.0
RAM 467024 467024 0 0.0
lighting-app debug+rpc+ui unknown 6104 6104 0 0.0
FLASH 5424673 5424673 0 0.0
RAM 204208 204208 0 0.0
lock-app debug unknown 5384 5384 0 0.0
FLASH 4681596 4681596 0 0.0
RAM 191216 191216 0 0.0
ota-provider-app debug unknown 4720 4720 0 0.0
FLASH 4299982 4299982 0 0.0
RAM 180184 180184 0 0.0
ota-requestor-app debug unknown 4672 4672 0 0.0
FLASH 4437358 4437358 0 0.0
RAM 184768 184768 0 0.0
shell debug unknown 4216 4216 0 0.0
FLASH 2950060 2950060 0 0.0
RAM 144040 144040 0 0.0
thermostat-no-ble arm64 unknown 9512 9512 0 0.0
FLASH 4039880 4039880 0 0.0
RAM 228528 228528 0 0.0
tv-app debug unknown 5712 5712 0 0.0
FLASH 5902693 5902693 0 0.0
RAM 590424 590424 0 0.0
tv-casting-app debug unknown 5288 5288 0 0.0
FLASH 11243517 11243517 0 0.0
RAM 710640 710640 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 907464 907464 0 0.0
RAM 142411 142411 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 901920 901920 0 0.0
RAM 124755 124755 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 846084 846084 0 0.0
RAM 141339 141339 0 0.0
nxp contact k32w0+release FLASH 584752 584752 0 0.0
RAM 70876 70876 0 0.0
mcxw71+release FLASH 600208 600208 0 0.0
RAM 63096 63096 0 0.0
light k32w0+release FLASH 611076 611076 0 0.0
RAM 70268 70268 0 0.0
k32w1+release FLASH 685512 685512 0 0.0
RAM 48680 48680 0 0.0
lock mcxw71+release FLASH 749024 749024 0 0.0
RAM 67500 67500 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1646756 1646756 0 0.0
RAM 211576 211576 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1553524 1553524 0 0.0
RAM 208392 208392 0 0.0
light cy8ckit_062s2_43012 FLASH 1439196 1439196 0 0.0
RAM 197144 197144 0 0.0
lock cy8ckit_062s2_43012 FLASH 1467412 1467412 0 0.0
RAM 224704 224704 0 0.0
qpg lighting-app qpg6105+debug FLASH 662340 662340 0 0.0
RAM 105220 105220 0 0.0
lock-app qpg6105+debug FLASH 620136 620136 0 0.0
RAM 99664 99664 0 0.0
stm32 light STM32WB5MM-DK FLASH 459736 459736 0 0.0
RAM 141568 141568 0 0.0
telink bridge-app tl7218x FLASH 665226 665226 0 0.0
RAM 90828 90828 0 0.0
contact-sensor-app tlsr9528a_retention FLASH 621908 621908 0 0.0
RAM 31488 31488 0 0.0
light-app-ota-shell-factory-data tl3218x FLASH 745418 745418 0 0.0
RAM 40496 40496 0 0.0
tl7218x FLASH 753974 753974 0 0.0
RAM 97632 97632 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 680678 680678 0 0.0
RAM 52192 52192 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 709240 709240 0 0.0
RAM 73400 73400 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 600796 600796 0 0.0
RAM 138912 138912 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 788928 788928 0 0.0
RAM 96488 96488 0 0.0
tizen all-clusters-app arm unknown 5104 5104 0 0.0
FLASH 1751708 1751708 0 0.0
RAM 93508 93508 0 0.0
chip-tool-ubsan arm unknown 11396 11396 0 0.0
FLASH 18683110 18683110 0 0.0
RAM 8181084 8181084 0 0.0

@yufengwangca
Copy link
Contributor

build_coverage.sh does not include clusters by default, you need to --code=clusters to include the clusters into the report

@mergify mergify bot merged commit 0a21983 into project-chip:master Feb 14, 2025
70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yufengwangca yufengwangca yufengwangca approved these changes

@mkardous-silabs mkardous-silabs mkardous-silabs approved these changes

Assignees
No one assigned
Labels
gn review - approved scripts
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@andy31415 @yufengwangca @mkardous-silabs @restyled-commits