Skip to content

Conversation

@briaugenreich
Copy link

Enable the balancer cost metrics to be included in the /jmx endpoint. This should be backwards compatible and has been deployed in our cdh5 fork for over a year.

@bbeaudreault
Copy link
Member

thanks! can you see about adding a test case somewhere? I think it'll be required in order ot upstream.

Also, can you format your commit message like this:

HubSpot Backport: HBASE-26745 Include Balancer Cost metrics in jmx endpoint

In general the upstream hbase requirement is to format like this: HBASE-26745 Include Balancer Cost metrics in jmx endpoint, with the leading jira id followed by jira title-ish. And then I've been following the convention of including HubSpot Backport in our own fork for stuff that will get upstreamed. The idea with that prefix is that it'd hopefully help us to resolve conflicts down the line since we know we can just drop the patch once the upstream is accepted

@briaugenreich
Copy link
Author

briaugenreich commented Feb 28, 2022

Validated with the additional metric attribute in the test. Running without the change produces the following test error:

[ERROR]   TestJSONBean.testJSONBeanValueTypes:118 expected:<{beans=[{name=test1:type=test2, 
modelerType=testClassName, intAttr=3.0, nanAttr=NaN, infinityAttr=Infinity, strAttr=aString, boolAttr=true", 
test:Attr=aString}]}> but was:<{beans=[{name=test1:type=test2, modelerType=testClassName, intAttr=3.0, nanAttr=NaN, 
infinityAttr=Infinity, strAttr=aString, boolAttr=true}]}>

@bbeaudreault
Copy link
Member

LGTM! Can you squash to one commit before merging? You can then cherry-pick that commit into a new branch based off upstream/master for submitting upstream.

@briaugenreich
Copy link
Author

briaugenreich commented Feb 28, 2022

LGTM! Can you squash to one commit before merging? You can then cherry-pick that commit into a new branch based off upstream/master for submitting upstream

Done! One quick clarification - I should merge this PR here and then make the upstream PR?

@bbeaudreault
Copy link
Member

Yep! It might require an addendum after the fact if they require changes, but so far that hasn't been a huge issue.

@briaugenreich briaugenreich merged commit c7e6f80 into hubspot-2 Mar 1, 2022
@briaugenreich
Copy link
Author

I just realized when I merged this in here it has the Merge pull request #4 from HubSpot/HBASE-26745-hubsp in the merge commit. Should i squash this merge commit into the actual work ?

image

@briaugenreich
Copy link
Author

Also opened my first upstream pr! 🎉

@bbeaudreault
Copy link
Member

Nice! We can leave the merge commit for now. Though in the future would choose "Squash and Merge"

I have the upstream PR an approval but we'll have to wait for a committer to take a look

@briaugenreich briaugenreich deleted the HBASE-26745-hubspot-2 branch March 1, 2022 14:45
@briaugenreich
Copy link
Author

@bbeaudreault Last question. Since this impacts the hmasters - should i still roll this out to all the hb2 cluster region servers with the upgrade job?

@bbeaudreault
Copy link
Member

probably not worth doing a full regionserver rollout for this change

bbeaudreault added a commit that referenced this pull request Jul 29, 2022
# This is the 1st commit message:

Add maven and RPM building in blazar

This uses our standard buildpack, but each module must get a custom .blazar.yaml to pass in required
properties to the build. One can distribute that .blazar.yaml using build-scripts/copy_blazar_yaml.sh.
In order to minimize the duplication, most of the customizations are consolidated in
build-scripts/prepare_environment.sh.

The RPM build is hooked into the standard build hbase-assembly, as a post-build step. The RPM build
scripts in hbase-assembly/rpm-build handle turning the assembled tar.gz into multiple RPMs. We required
some minor changes to the descriptor in hbase-assembly to account for the fact that our build processes
each module independently instead of as a single reactor build. As a result, moduleSet does not work, so
moduleSet usages were converted to dependencySet.

# This is the commit message #2:

Fixes to get HubSpot build to work

- Our branch version munging script set-maven-versions can't parse the pom.xml due to unicode, remove it.
- Our build adds build.json files to the jars. Allowlist those files in jar content verification.

We could potentially try to upstream generic versions changes of these at some point

# This is the commit message #3:

Add client bundles for hbase2 (#11)

* Add hubspot client bundles

* add cdh5 mode and build the client bundles using cdh5 hadoop and java8 for now

# This is the commit message #4:

Add AggregationClient to client bundle (#12)


# This is the commit message #5:

Shade hadoop compat classes so they can be loaded in hadoop jobs (#13)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants