Added a method to create a mysql database from a connection object by Seb-C · Pull Request #583 · golang-migrate/migrate

Seb-C · 2021-06-17T07:54:06Z

This PR implements #578 by adding the WithConnection method to the MySQL driver.

I could test and verify that it works. However, I don't have the environment or time to provide and test the same change on all other drivers, so it creates a small inconsistency. How do you want to proceed?

coveralls · 2021-06-17T08:26:25Z

Pull Request Test Coverage Report for Build 492

24 of 28 (85.71%) changed or added relevant lines in 1 file are covered.
578 unchanged lines in 14 files lost coverage.
Overall coverage increased (+0.3%) to 57.462%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
database/mysql/mysql.go	24	28	85.71%

Files with Coverage Reduction	New Missed Lines	%
database/stub/stub.go	6	71.93%
database/ql/ql.go	25	62.65%
database/sqlcipher/sqlcipher.go	25	66.48%
database/sqlite3/sqlite3.go	25	66.48%
database/sqlite/sqlite.go	25	66.48%
database/firebird/firebird.go	30	75.3%
database/cassandra/cassandra.go	42	45.89%
database/redshift/redshift.go	45	67.43%
database/sqlserver/sqlserver.go	47	67.76%
database/mongodb/mongodb.go	55	73.83%

Totals
Change from base Build 468:	0.3%
Covered Lines:	3623
Relevant Lines:	6305

💛 - Coveralls

dhui

Thanks for the PR!

dhui · 2021-06-23T18:43:19Z

 	return mx, nil
 }

+func (m *Mysql) setupDefaultConfig() error {


Can you move all of this logic to WithConnection(). Then inside WithInstance() we can call WithConnection() and set m.db = instance.

Aha, I considered both designs and I seem to have picked the wrong one 😭
I'll change this ASAP, thanks for your review.

Done!
By the way, I just reminded why I chose the previous pattern: It's because now I need WithConnection to return a *Mysql type instead of database.Driver.

Why do you need a *Mysql?

Because WithInstance needs to set mx.db

dhui · 2021-06-25T22:28:15Z

I could test and verify that it works. However, I don't have the environment or time to provide and test the same change on all other drivers, so it creates a small inconsistency. How do you want to proceed?

This is fine. We can add WithConnection() to other DB drivers as needed/requested.

dhui

Thanks for addressing my feedback!

dhui · 2021-06-29T17:25:09Z

+
+// instance must have `multiStatements` set to true
+func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
+	if err := instance.Ping(); err != nil {


Move this to WithConnection() and use conn.PingContext(). DB.Ping() will create a connection to ping anyways

The problem I am trying to solve is that this class does unexpected stuff to my connection pool.
If I inject an active pool with n idle connections to this class, do my stuff and close it, I expect my pool to be returned to it's initial state, which means an active pool with n idle connections.
But currently I can only choose between ending-up with a closed pool (because Close not only returns the connection to the pool, but also closes the pool itself), or (if I don't call Close) a pool with n-1 idle connections and 1 allocated connection that will be stuck forever

So the last thing I want is for WithConnection to create new connections in a pool that I did not even inject in the first place.

If the connection I inject is not working, I expect the process to fail without side-effects on my pool.

If the connection is not working, doing conn.PingContext().DB.Ping() would create a new connection in the pool and ping the new connection, which does not test or fix the injected one anyway.

My understanding is that connections are pooled at the sql.DB instance level so conn.PingContext() shouldn't create a new connection e.g. connections won't create new connections

Sorry, I was misunderstanding your question.
I added the context to WithConnection, as well as calling PingContext.
I cannot add it as a parameter in WithInstance without breaking change, so it should be done along with #14.

dhui · 2021-06-29T17:34:15Z

-// instance must have `multiStatements` set to true
-func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
+// connection instance must have `multiStatements` set to true
+func WithConnection(conn *sql.Conn, config *Config) (*Mysql, error) {


Thoughts on making this context aware? Eventually, we want most DB driver methods to be context aware. I'm thinking that new methods should be context aware.

Also, unfortunately, these functions that create DB drivers currently return interfaces instead of concrete times. I'm thinking returning a concrete type as you're doing now is a cleaner pattern. Can you add a type check? e.g. var _ database.Driver = (*Mysql)(nil) // explicit compile time type check

If it's consistent across all drivers, then it would be a good thing. But it seems out of scope for this PR.

Can you add a type check?

Sorry, I don't understand what you are asking. Where do you want this check?

If it's consistent across all drivers, then it would be a good thing. But it seems out of scope for this PR.

Oh yeah, making the db drivers consistent is not in the scope of the PR. But new code should fit within the future plans/vision if possible. In this case, we should add a ctx context.Context parameter to WithConnection()

Can you add a type check?

Sorry, I don't understand what you are asking. Where do you want this check?

Add var _ database.Driver = (*Mysql)(nil) // explicit compile time type check as a package level ignored variable.

OK, I understood your point.

Done (both of it).

dhui

Thanks again for the PR and for addressing my feedback!

Seb-C · 2021-07-08T03:02:35Z

@dhui Thanks for merging my PR 🎉
Do you have any timeline about the next version release?
I need to plan if I reference the commit hash in my go.mod file or wait for the v4.14.2 being released.

dhui · 2021-07-08T04:10:10Z

Release are blocked, so use the commit hash

dhui · 2021-08-06T18:20:24Z

v4.15.0-beta.1 was released recently. v4.15.0 to follow soon.

Added a method to create a mysql database from a connection object

f98ad3a

dhui requested changes Jun 23, 2021

View reviewed changes

Calling WithConnection from WithInstance to de-duplicate code

394279a

Seb-C force-pushed the mysqlWithConnection branch from 1b8c309 to 394279a Compare June 29, 2021 07:30

dhui requested changes Jun 29, 2021

View reviewed changes

dhui mentioned this pull request Jun 29, 2021

New Driver Interface #14

Open

Sébastien CAPARROS added 2 commits July 6, 2021 10:16

Adding context and ping to mysql.WithConnection

73081e2

Interface type check at compile time

4bfe192

dhui approved these changes Jul 6, 2021

View reviewed changes

dhui merged commit 31cedbb into golang-migrate:master Jul 6, 2021

Fontinalis mentioned this pull request Aug 23, 2021

Database driver closes the connection pool #578

Closed

AndreasKl mentioned this pull request Nov 29, 2021

Add WithConnection to Postgres similar to MySQL. #659

Merged

selaux mentioned this pull request Aug 16, 2022

Implement WithConnection for sqlserver database driver #794

Closed

jtwatson mentioned this pull request Apr 2, 2023

feat(spanner): Add option to disable closing spanner clients #910

Closed

bclarkx2 mentioned this pull request Sep 21, 2023

Add WithConnection for pgx and pgx5 nicheinc/migrate#2

Merged

bclarkx2 mentioned this pull request Sep 29, 2023

Implement WithConnection for pgx and pgx5 database drivers #982

Open

Conversation

Seb-C commented Jun 17, 2021

Uh oh!

coveralls commented Jun 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 492

💛 - Coveralls

Uh oh!

dhui left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dhui commented Jun 25, 2021

Uh oh!

dhui left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Seb-C Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dhui Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Seb-C Jun 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dhui left a comment

Choose a reason for hiding this comment

Uh oh!

Seb-C commented Jul 8, 2021

Uh oh!

dhui commented Jul 8, 2021

Uh oh!

dhui commented Aug 6, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coveralls commented Jun 17, 2021 •

edited

Loading

Seb-C Jun 30, 2021 •

edited

Loading

dhui Jun 30, 2021 •

edited

Loading

Seb-C Jun 30, 2021 •

edited

Loading