Learnings of the Week
End of April #
Well it's already the start of May, and it's kind of crazy how quickly the
last month went on. Definitely hope to have more interesting things learn in
the weeks to come.
patch() and passing tests #
So in a previous blog post, I talked about how I used patch() in test
methods in order to bypass how the production code calls databases, and how
the test code creates test databases. All tests passing means that everything
is good right? Wrong. Passing tests are only passing for what you tell it to
be passing, and if you're not testing in the proper procedure, you might end
up with a passing test, and then a failing code run. That is precisely what
happened with me, after I had thought that the tests I wrote were done.
The problem was that the routine I was testing made calls to a production
database in several different which when testing, I didn't care for. So of
course when doing another functional test from the command line, it failed.
This had me scratching my head for a second because all the tests passed
right? The system went from a state where everything was working, then I wrote
tests, and tests were green, then the system stopped working. It is very
important to break down the situation and to set back and re-evaluate what
could have gone wrong. Luckily I had a good feeling that it had to do with how
I changed the calling and connecting of the production database, and the
problem was solved. But after all this, I still think there is lots of room
for refactoring of how database connections are managed within an application,
and how to properly design the different modules such that testing is more
effective.
Weird Postgres network issues #
So this isn't really something I have finished learning, but with one of my
deployed applications, I have been running into intermittent SSL SYSCALL ERROR messages that pop up, and are pretty much un-reproduceable. The errors
occur, and then fix themselves almost immediately, so a simple refresh usually
solves the problem. The main issue I have with debugging so far is that I
cannot get this to happen in my local development, and the errors only occur
in production. So the main difference between the two is that the production
version talks with databases hosted on AWS RDS. This makes me think that the
errors have something to do with the network connection between the two, or
perhaps something to do with how the Postgres database is set up in RDS.
I've spent some time trying to diagnose whether or not this bug is because of
the code I've written, which is very likely since this is my first
implementation of SQLAlchemy. But I've noticed that I get this drop of
connection after some time in psql as well… My next steps are to look at the
pg stat activity and understand more about what the connection messages
mean. It would very well be how the connections to the database are managed
within Postgres, and not cleaned up accordingly.
Postgres RTFM #
So the Postgres manual is over 2000 pages, I guess it's time for me to RTFM.