T-SQL Tuesday #110 – “Automate All the Things”


This month is being hosted by Garry Bargsley – thanks @gbargsley!

My response to this month’s topic is a personal one. The main two questions to answer are:

  1. What do you want to automate or what automation are you proud of completing?
  2. What is your go-to technology for automation?

This year my goal is to work on a double program, data science and artificial intelligence. I have chosen the Python path (or it chose me). So Python is becoming my go-to hammer in search of nails so I am all about, “Automate With Python All The Things”.

A book I am working through as I am learning Python is called, oddly enough, Automate the Boring Stuff With Python. LOTS of things I could automate that I had no idea about. Highly recommended.

Specifically, I am interested in Chapter 17, image manipulation.

Do You Take Pictures?

Please raise your hand if you take pictures. I see everyone’s hand went up. Yes, with digital cameras and smartphones, we take a plethora of pictures. Some would say too many! We don’t even take time to go back and look at all of them. Storage is so cheap we can be digital hoarders and not feel ashamed.

How Do You Backup Your Pictures?

Um, ya. About that. As data professionals we should know the importance of backups and having a restore strategy. Our backups should be local for immediate retrieval and off-site too. For personal use, backups should be off-site in some kind of cloud storage.

But how many of us regularly and with discipline and/or automation backup our pictures off of our smartphones, digital cameras, laptops and desktops? And is it a pure file-based backup or do you put pictures into a database?

Is Putting Pictures Into a Database Overkill?

Some would argue yes, that’s silly. Others would say no but how to do it? And what would the benefits be to storing images in a database?

Previously Done in R

I read this article a while back and worked through all of the examples. This was before I was bitten by Python. So I want to re-do all of this but do it in Python and store images in a database.

A Gift to My Wife

Chris and I have been married 25 years. As my gift to her, I am building an art gallery of images from the last 25 years. So far I am up to 300 images. Now, this isn’t a slide show. It is going to be a gallery built using Unreal Engine and deployed onto an Oculus Go.

But in order to properly size the images, they need to be a power of 2 as in 2048×2048. So! How do I fiddle with 300 images? Re-size, extract meta-data, store in a database, etc?

Enter Stage Right – Pillow and Scikit-image

Pillow (used in the book above) and scikit-image are both image processing packages.  Ever wonder how to count the number of marbles in a picture? Facial recognition software? These packages are amazing and they have a whole lot of really neat and cool capabilities. While some of the scientific stuff is over my head, I did pick up some things from this course- but I will need to re-take it again once I understand things better during the study of data science. Building Image Processing Applications Using scikit-image

SQL Saturday Topic

It shouldn’t be a surprise then that manipulating and storing images in a database is going to be my new SQL Saturday topic/theme for this year. It does sound like total overkill or using a cannon to shoot a fly. Some might say that is just good target practice. The technologies I am working with does sound like a lot and it is: Hekaton, FileTable, Python, Unreal Engine, and Virtual Reality for starters.

And before you ask, yes, I will be sharing all of my scripts and who knows, might even create my first community tool 🙂


This entry was posted in Uncategorized. Bookmark the permalink.

1 Response to T-SQL Tuesday #110 – “Automate All the Things”

  1. Kris says:

    Bit of a tangent in case of interest:

    >> As my gift to her, I am building an art gallery of images from the last 25 years …

    We have a Digital Photo Frame in our hallway. At the end of each year I select a small handful of photos taken during the previous year and add them to the USB stick in the Photo Frame. Whenever we walk through the hall it is great to catch a glimpse of a long-since-forgotten photo.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s