Wow, we had a variety of responses to the April 2019 topic of “What Do YOU Use Databases For?”
I think the overall response to the question and the theme is both mixed and varied.
I have been struggling with the personal use of databases for a long time. Things I wish would have been easier but seems to just get more complicated over time. Ever heard of GDPR? Although we think we have absolute control and access to data about ourselves, we really do not. The right to be forgotten is NOT the same as having access to all of the data about ourselves in all of the systems before they disappear. Sometimes companies will delete your data about you before you ask.
Case in point: Microsoft HealthVault. It is going to die. And your data? Deleted. https://en.wikipedia.org/wiki/Microsoft_HealthVault
Tell me- if your personal physician or other health care practitioner wanted to access your HealthVault records 20 years into the future, how would they do it? After the service has long been terminated and your digital data destroyed? They cannot. Did you pull out your data before the service was terminated and put it someplace else?
Do you know who is responsible for digital data about you? It is the same answer as to who is responsible for your own data career.
YOU.
We are like a Stranger Living in a Strange Land (Yes, I am tainted by Heavy Metal. It is a part of me. And no, I am NOT ashamed. https://www.youtube.com/watch?v=UJsl-bB7lmk).
Digital records are easily created, changed, and deleted. We are living in a so-called enlightened and evolved age compared to the luddites who embraced physical paper records of data. A short time ago, I posted via Twitter my concerns about Wills and Testaments in this digital age and how often do people perform DR tests to access to our digital records, especially after we die…
We believe we have access and control to all of the data collected about ourselves but we do not.
In an intentional manner, this month’s T-SQL Tuesday topic was just that- when we choose to create databases for ourselves, but what do we do or use it for and why? How do we back it up?
The crazy part is that for the ubiquitous SQL Server .bak file we now have to worry about having access to the versions of the software to restore it. Today I can open up .pdf files from over twenty years ago. Can one do that with .bak files?
No.
I have to restore the .bak to an old version of SQL Server, then back it up again using a relatively newer but compatible version, then restore it again using a newer version of the software, rinse and repeat until I can see the data in the latest version of SQL Server. Cumbersome to say the least.
Believe me when I say there are non-technical people who don’t want to have to fiddle around with ANY of this stuff- they just want it to work and have some kind of guarantee that some time in the distant future their digital files will be able to be successfully opened and used by people they want to be able to open and use the files.
We use databases all of the time and dare I say it? (Dare, Dare!!!) Databases use US. Ever wonder how when you click on something at work and then you start seeing ads for things on your personal devices? Hm…
For example, why would you build a database schema and then both build and populate data via a UI when one could simple use an existing service to track your finances or fitness data? Several of the responses to this month’s post seem to echo that sentiment- why build when you can buy? However some folks do it anyway (like me) as a learning experience much like I did with our VHS collection. People are willing to pay good money to make pain go away quickly and efficiently. They don’t want things to drag out over time. Instant gratification to a problem via an app or existing solution trumps wasting time (!) building something from scratch. But alas, like the demise of Microsoft HealthVault, is one willing to sacrifice inputting today’s data into a system that could both die and not be able to extract one’s data out of it sometime in the future?
I keep going back to both the book and the project MyLifeBits. It rings a deep chord with me. I imagine it will continue to do so. We should be able to have access to ALL of our own personal digital data and decide what happens to it over time. When we control the data ourselves via our own schemas and systems, that becomes fairly straight-forward and yet is time-consuming. But when we outsource that data and storage we relinquish control and trust and hope it will be available to us or loved ones sometime in the future.
Without further ado, here are my observations and comments on this month’s T-SQL Tuesday topic:
Jeff talks about the skills we need to stay sharp by being able to easily spin up a virtual machine and jump right in. (On a side note, I got an e-mail just yesterday thanking me for creating a document to build a virtual lab for the 70-462 (SQL2012) exam. Time flies). There is a lot of value in understanding the entire stack in a personal application. It is just enough complexity to wrap your brain around but not too complex that one feels overwhelmed. Building and re-building hones our skills and provides insights into troubleshooting. Keeping an eye on income and expenses via spreadsheets is smart but ideally still need to maintain security and keep it backed up somewhere out of the home. He ends by learning to play with data by importing and manipulating large datasets. Thanks Jeff!
Kenneth Fisher (@sqlstudent144 | b)
Kenneth reveals he likes designing and building crossword puzzles using SQL Server. Awesome! More than one person on this topic has referred to ERD(s). (BTW, whenever I am at a client and before I start poking around with DMV(s), I like to restore a backup of a database and create an ERD. It provides a way to visually see tables and relationships. It also can show lack of relationships. NOTE!!! This can take a while using Database Diagrams in SSMS. One observation over the years- the tables with the most number of constraint connections to them are some of the most used tables in queries. Which tables, indexes and queries to tune first? The ones which are used the most.) The entire Head First series from O’Reilly has crossword puzzles at the end of every chapter. It is part of their metacognitive strategy to writing these books. It is a great way to re-enforce concepts and gets you thinking about things in a fun way. Now I’m thinking about how to build a 3D crossword puzzle in virtual reality so one could walk/fly around it. I like word art and shadows. Thanks Kenneth!
Kevin discusses the future of smartphone apps hitting databases in the cloud. Anybody with a smartphone is constantly generating CRUD queries. Ever think about how many queries are run by you in a 24 hour period via any and all apps? And the growth of data stored in the cloud is going to continue to grow. Thanks Kevin!
Tracy Boggiano (@TracyBoggiano | b)
Tracy shares she has a sports card collection database with over 50,000 cards! That’s cool. I’m curious how the pricing gets updated and captured- can you see the delta(s) of price changes over time? And is it a manual process or automatic? Interesting. It is encouraging to see people getting into databases at a very young age and being able to create something useful for themselves! She also captures fitness data. I’ve seen many people wearing fitness trackers over the years. Most smartphones also have some kind of default fitness app which uses the phones accelerometer too. Being able to quantify your health helps to keep one mindful about your current and future health goals. As the saying goes, if you can quantify it, you can measure it. Thanks Tracy!
Reitse Eskens (@2meterDBA | b)
Reitse talks about volunteering in speedskating. And the challenges of having to sync two applications for the data. Getting Windows Updates during a race, ARGH. (In the past I used to help out with Cub Scouts and AWANA Pinewood Derby races (sensor on release and on finish line) and different kinds of races. Generally each car will race once down each lane. So the software figures out how to organize all the racers. It had a Microsoft Access database back-end on a laptop). People are storing more and more personal data in the cloud but I think each individual needs to decide what works best for them. I am uploading a ton of .zip files of photos to my OneDrive and it….is…slow… And I like your tractor analogy too. Thanks Reitse!
Jason echoes others in use of personal databases to trace fitness and finances. I really like the idea of dreaming up small things to try and test and do all over again. His link to Steve’s post is spot on- I would add this applies to speakers in the data community as well. Do you know how hard it is to convince people to present? Our community is filled with a lot of really smart and experienced people but a lot of them have no interest in public speaking. They are either not comfortable with it or don’t see any benefit. To be blunt: if we are not learning something new, we are slowly dying in our craft. He ends with a call to use automation (including inside a database) to sustain your work and I would add your sanity. I’m still slugging through a Python book called Automate the Boring Stuff With Python as a way to a) force myself to learn a new language and b) learn something besides T-SQL to do useful work. Thanks Jason!
Eugene has played A LOT of games. Games always have been a social experience played with others. He does show something that is prevalent in ALL data-entry systems- quality and time to perform data-entry. I can’t tell you how many times I’ve see free-form fields used in applications and then people wonder where did we get all of this dirty data from? Can we clean it up without breaking things??? I like how they play video games together too and that is important. And if kids are in anybody’s future, I would advise ALL of you to play both board and video games together. It creates some of the best memories one will have for life. Thanks Eugene!
Andy Levy (@ALevyInROC | b)
Andy is into geocaching using GPS devices wherever he goes. (NOTE!!! My twenty year old Magellan GPS stopped working on April 6th. It is now a museum piece). If I had to find a fake pinecone and money was no object, I might spring for a FLIR for my phone. Hiding in plain sight, I love it. You should see the number of caches he has found and some of the metrics, wow! I want to get back into it but need to be aware of rattlesnakes where I live. More ERD diagrams like some of the other posts but this time from reverse engineering a SQLite database and then doing some PowerShell ETL into SQL Server. This is great practice for those considering getting into IoT devices too. Great stuff and a great excuse to get outside and out from behind our computers. Thanks Andy!
Nate Johnson (@natethedba | b)
Nate makes the case that he’ll get his hands dirty when he needs to but for simple things like an inventory list, just use a simple tool to get the job done in a short amount of time like Google Sheets. There seems to be an ever-present pressure to try to be on the latest versions or playing around with the latest technology for SQL Server. Personally I still owe a review for a Linux on SQL Server book that I’ll get around to probably by summer. I work in a Windows shop and all my home systems are Windows. But I do want to learn about Linux and containers especially after I’ve seen Bob Ward present on them. Magic! Just not a super priority for me right now. Physical to virtual machines is the first step before learning how to migrate to the cloud. Thanks Nate!
Lisa Griffin Bohm (@LisaGB_sql | b)
Lisa started using databases to help plan her first wedding! It is a lot stuff to track and you don’t want to risk hurting people’s feelings by forgetting something. Saving money by doing it yourself when it makes sense is smart. At least for address labels for Christmas cards, I have used Word’s mail merge program and Avery labels. More and more people just want to keep things simple like using Trello. Thanks Lisa!
Michelle Haarhues (@mhaarhues | b)
Michelle is an avid runner and loves to go on destination runs! While some apps do a great job of tracking runs, they don’t have an easy way to add additional things like destinations to see or store advice about items like health stats or run suggestions by friends. Moving from a spreadsheet to a database is not always easy. A training partner database has a nice ring to it. Being able to combine vacationing and running sounds like a great way to stay in shape and keep learning databases. Thanks Michelle!
Codename “BonzaOwl” needed to track vegan-friendly items electronically like in a database. When one is passionate about the data it takes on a certain level of importance. Being able to track food choices is critical for several people I know; there were some kids in my son’s Boy Scout Troop years ago that had peanut allergies. No joke- he could have an allergic reaction to something that got cross-contaminated and if we were in the back-country on a camping trip, professional medical aid might not be easily available even with an EpiPen handy. It is great to see people creating databases for themselves for product choices like this. Thanks BonzaOwl!
Rob Farley (@rob_farley | b)
Rob makes the astute observation that for a majority of his clients and personal needs, it is better to buy than to build a solution when you can afford it. Even smartphone apps have come a long way- it seems like there is an app for everything if you know what one is looking for. He has been in the data industry a long time and values both his own and his client’s time. Thanks Rob!
Steve Jones (@way0utwest | b)
And last but certainly not least, Steve talks about work in progress items across different fields like bucket lists, speaking events, and fitness data. Even SQL Saturday stuff! It seems like writing a lot of T-SQL to do data-entry is cumbersome from many of the posts. While I haven’t used Microsoft Access in ages, I do hear a bit of a resurgence in its use for simple client/server applications like hitting a SQL Database in the cloud. Steve and I have chatted about the MyLifeBits Microsoft Research project in the past and dealing with many disparate sources of data today for personal use is all over the place. Structured and unstructured data lives in many different places- and some of those places go out of business!
In Summary
I wanted to thank Steve for letting me host. This was my first time hosting. And Thank You everyone! Your participation has given me some great insights and ideas. –Todd
Todd Kleinhans 
Pingback: T-SQL Tuesday #113 – What Do You Use Databases For? – T-SQL Tuesday
Pingback: T-SQL Tuesday 113 Roundup – Curated SQL