T-SQL Tuesday #137: The Best of Both Worlds – RAPIDS and SQL Server

During Lent 2021, I stepped away from writing and social media. Winter is over and I’m looking forward to a lot of writing in the months ahead.

This month’s T-SQL topic by Steve Jones is on using Jupyter Notebooks. I first saw their use for the first time at the PASS conference in 2019 (remember PASS?). What I thought was cool was the ability to both run code and save results too. Even have text, images, etc- just like a paper notebook.

I’ve been a SQL Server DBA for most of my data career. Some of you have wondered what the future looks like for DBA(s) and something I’ve wanted to have a reason to get into is data science. I don’t remember exactly how I first heard about RAPIDS from NVIDIA, it was probably something I stumbled across from following them on Twitter.

NOTE!!! Don’t miss the free conference starting on 12Apr21 all week, they will have a lot of new announcements and plenty of free training resources too: GTC.

What Is RAPIDS?

From their website, “The RAPIDS suite of open source software libraries and APIs gives you the ability to execute end-to-end data science and analytics pipelines entirely on GPUs. Licensed under Apache 2.0, RAPIDS is incubated by NVIDIA® based on extensive hardware and data science experience. RAPIDS utilizes NVIDIA CUDA® primitives for low-level compute optimization, and exposes GPU parallelism and high-bandwidth memory speed through user-friendly Python interfaces.”

The teams at NVIDIA have been working really hard to try to maintain a familiar Python syntax for existing libraries but have them run on the GPU(s). Here is their release roadmap; version 0.18 is what I’m using.

GPU stands for Graphical Processing Unit. Anybody with an NVIDIA graphics card (Pascal or higher) can run RAPIDS. Right now, it only runs on Linux; I failed to get RAPIDS to use Windows Subsystem on Linux (WSL) as I only have a Maxwell based GPU on my gaming rig. I’m hopeful that once I get a new video card, I’ll re-visit running RAPIDS on WSL. If you are wondering, I tried to follow the steps both here and here.

Here is a video talking about how Wal-Mart is using RAPIDS.

Dual Boot Using An External SSD Drive

It just so happens I received a new laptop for work and it has a Pascal GPU! But alas, this is a work system so I can’t install Windows Insider Preview versions. I also didn’t want to risk fiddling with partitioning the internal drives either.

Low and behold! I stumbling across this video trying to research installing Ubuntu on an external drive. Since I had all of the parts, I didn’t have to buy anything. NOTE!!! The video is over a year old and some of the steps have changed. Just be sure to stick with Ubuntu 18.04; don’t upgrade to 20.04.

I have been running this for a few weeks now without any problems. My work laptop is unaffected and I can learn RAPIDS using this configuration albeit with a limited amount of GPU memory.

Installing SQL Server on Linux

Sticking with the instructions for Ubuntu 18.04, I installed SQL Server on Linux. I also got Azure Data Studio for Linux installed too. Downloaded and restored the AdventureWorks databases. I then ran through several steps to get the Python drivers installed. This site was very helpful (remember to stick to the 18.04 instructions) and I was able to get things running from within a Jupyter notebook. Finally!

Running Jupyter Notebooks Using Docker and RAPIDS Container

Click on the link [http://localhost:8888] in a browser like Firefox and the notebook will open.

Since I’m focused on leveraging existing data skills to learn GPU things, I was interested in things that had a SQL syntax. Note there are several notebooks covering many of the tools within RAPIDS which can be found in the container.

(caveat: the following examples are from https://app.blazingsql.com which will be shutting down and will be re-branded as something else? Time will tell.)

What follows are several screenshots from me copying code from their site and running it locally on my machine. These examples use NY taxi data (46MB).

What is so neat and cool to me is to be able to run queries and plot data without a whole lot of headache or software installation.

SQL Server -> Pandas -> cuDF

The component known as BlazingSQL is an in-memory data analytics tool which uses the CUDA dataframe (cuDF)- it doesn’t persist data. By using SQL Server, one can use all of the tools many of us are familiar with. The following screenshots show how I connect to SQL Server on Linux using Python inside of a notebook and then load a GPU dataframe, then show some metrics about it.

These are the results from running gdf.info(verbose=True) from above:

Conclusion

I know this was a whirlwind tour. Jupyter notebooks has made it easier to query data, see results, and do Pythonic stuff. RAPIDS is getting a lot of attention as it can do things much faster on the GPU than on the CPU like SQL Server. I hope to write more about RAPIDS and its use in data science.

Posted in Uncategorized | Leave a comment

T-SQL Tuesday #134: Stopping to Breathe Again and Talking Avatars

The first post of 2021 for T-SQL Tuesday is being hosted by James McGillivray and his updated post on the subject of taking a break can be found here. Thanks James!

Sadly one of the members of the #SQLFamily has passed away due to complications from COVID-19, Gareth Swanepoel, and the organization once known as PASS will be no more on 15Jan21.

I maintain #SQLMemorial and you can watch the latest tribute video to the fallen here.

Week 1: A Fresh Start

I’ve written Part 1 and Part 2 about getting into MBSR so this is more of a re-start than a Part 3 post. When the pandemic really kicked into high gear mid-March 2020, I just didn’t stick with the commitment of 40-45 minutes each day for eight weeks. A few weeks into it and I spent my time getting outside to go for walks and taking power naps instead.

2021 is a new year with new goals so I’m stopping to breathe again. Secret? Put it on your calendar throughout the day and make time. All of the PDF(s) and audio files for the MBSR course I’m going through can be found here.

The Good, The Bad, and The Super Scary (Or Not)

I have about 100 pages left of a book I’m reading, VRx: How Virtual Therapeutics Will Revolutionize Medicine by Dr. Brennan Spiegel.

It is a fascinating book and I’m really enjoying it. In one of the chapters, he lays out a exploration into the treatment of augmenting an existing therapy using VR for people with schizophrenia which grabbed my attention.

Back in March 2019, I wrote a little about digital avatars and the incredible progress in creating live-like avatars but using something other than a human. T-SQL Tuesday #112 “A New Cookie Jar”.

Now this technology is becoming more accessible. Take a look at the beta version of Audio2Face video created as part of NVIDIA’s Omniverse suite of software. Please note that it is only using an audio file- no programming by the end user was required.

Back to VRx. Imagine someone being bullied by their inner demon, a satanic figure with two ram-like horns, blood-red skin, massive bat wings, and talking to you a short distance from your face. “You are no good. You are a bad father, You are a bad husband. You are worthless!” Chiding, insulting, distracting, incessantly.

Disturbing visual and audio to say the least. But that is what a psychiatrist has been able to do- create an avatar as close to his patient’s hallucination but using the psychiatrist’s voice. And over the course of treatments, the audio becomes, “You’re not as bad as I thought. I underestimated you.”

So, let’s look at the flip side for positive mental health applications. Imagine someone or something saying positive, uplifting and affirming quotes and sayings. How would that make you feel? Better than trying to talk to oneself in a mirror I suppose.

While positive psychology entails more than just technologies, it could be interesting times ahead for positive avatars helping and encouraging people.

One More Thing

I know several folks who are in the #SQLFamily and are into video games, myself included. Epic Games releases a FREE game every week from their on-line store.

On 14Jan21 for all of you Star Wars fans, Star Wars: Battlefront 2: Celebration Edition will be released and free for a week. Once you claim it, it is yours. I have over 120 games I have downloaded from the store over the past two years- all for free.

Posted in Uncategorized | Leave a comment

The 2020 Fall Unreal Engine Learning Challenge – Lessons Learned

Introduction

I have been into Unreal Engine as a hobby for a while, and I’m not a game developer by trade or by education. When learn.unrealengine.com came out, I was excited to take courses.

On November 18, 2020, they announced the 2020 Fall Unreal Online Learning Challenge and I decided to jump in. Truth be told, I had already taken the Build a Detective Office Game Environment course but I took it again anyway as a refresher.

I hope this Lessons Learned post will inspire people to jump into Unreal Engine now and in the future as it is a fantastic game engine but thanks to this FREE learning portal, it is not as intimidating to learn as it once was. As the saying goes, if I can do it, anyone can do it!

About Me

I have been working with databases as a full-time job for over 20 years and currently work for a non-profit. I help run the Denver/Boulder Unreal Engine Group and thought this would be something fun to do- to lead by example and dive right in. Someday I really want to get back to meeting in person (I’m in Zoom calls all day long). I use UE4 for a lot of personal projects and am looking to do even more with it in 2021. Stay tuned!

Attack Plan

In order to take all five courses in the time allotted, I really needed to focus and set aside other tasks so I could concentrate and complete on-time. My methods are as follows; they may or may not work for you but they work well for me.

I use OneNote for screenshots of slides presented in addition to stopping the videos and taking notes.

The authors of these courses are very knowledgeable and I don’t want to miss anything.

Second, and this is extremely time-consuming, I watch ALL of the videos, straight through, and only stopping and taking notes if it rings faint bells and I need to research something further. Meaning, I just want to “see” the whole thing in a short period of time to get a feel for what I’m getting myself into.

Third, I re-watch them, going slow and taking notes as I go along and working through all of the examples while I have UE4 open and following step by step. If one were looking over my shoulder, one would see me using two screens, OneNote, UE4, SnippingTool on one, and the course itself on the other. I’m constantly cycling through windows using Alt+tab as I’m going through the material.

In my experience so far, I average about 2-3 hours (sometimes longer if I’m not getting or immediately understanding everything) for EVERY hour of content. So a three hour course might take me all day using this method.

One more thing which I find extremely important- REVIEW!!! If one is taking the time to laboriously take so many notes, take time to review them immediately afterwards since writing them, and glance at them before starting the next module. Try to recall (from memory) what the content was about and your own notes on them. This is hard and yes, even boring at times- new content to learn is calling!!! However if one really wants to understand the material, slow down and take time to review. It is an investment, and not a waste of time.

I do try to follow the authors on Twitter if they are on there (I’m @toddkleinhans by the way) and add any and all links to books, courses, other resources they recommend. Note that some of the courses they reference may or may not have already been published.

Other Thoughts and Observations

4.26 is the latest version as of this blog post on 13Dec20. It is the nature of software development to always be upgrading. I had to install 4.23 to complete some of these and it felt quaint 🙂 Always use the version the authors’ are using as this will help to minimize the risk of the projects files not working correctly.

I wish the courses had transcripts with them- I spent A LOT of time stopping and starting the videos over and over again just to catch and type what the author(s) were saying.

Learning about the organization and naming system for things was valuable too.

I LOVED the deep-dive into The Village for some of the courses. An exploration into the look, feel, mechanics, Blueprints and C++ code behind the game. Being able to “wrap my brain” around it was important and highly instructive to me.

Even though I have successfully completed the Challenge, I can’t wait to take other courses in 2021 too. I not only want to see so many more badges on my Dashboard, but more importantly, I want to be able to apply what I have learned. And re-visit the courses when necessary- don’t be embarrassed if you forget something or need a re-fresher!

Personal Project

After completing this Challenge, I now have the technical insights and expertise to re-visit and improve a personal project of mine.

My first blog post on it can be found here.

Some of the technical aspects can be found here. Note that I now know how NOT to do things in the Level Blueprints, I’ll be changing and improving it, based on the audio courses I just took. Heh, I did have that overwhelming feeling of, “Oh… I didn’t know that… I need to change that when I get a chance!!!”

My latest YouTube version (using OBS to screen record audio and video) here.

Conclusion

Keep taking courses, notes, and adding to one’s own toolbox. This Challenge was intense for me. It really gave me a good proverbial kick in the pants to step up and dive in while yes, at times, being COMPLETELY outside of my comfort zone but slogging through, re-watching things over and over again until I “got” it.

The Challenge opened my eyes to several new things and many ways to improve existing projects I am working on. Thanks for reading!

Posted in Uncategorized | Leave a comment

T-SQL Tuesday #133: What (Else) Have You Learned from Presenting?

For the T-SQL Tuesday topic this month, Lisa Bohm asks, “This month, I’d like those of you who have presented, or written a presentation, to share something technical THAT DID NOT RELATE to the topic of the presentation, that you’ve learned in writing or giving the presentation.”

I had an idea of how to do a better technical presentation years ago using a video game engine instead of PowerPoint. Why? I could “walk around” inside of a game. It had so many parallels to memory palaces. But it was hard for me and it just seemed like I was trying to wield a hammer in a blind search for nails.

From walking around execution plans, to various joins and seeing visual intersections of spheres of data, I just wanted to go wild with a sight and sound exploration of technical topics using intense video game capabilities.

“As a doctor, you of all people should be aware of the dangers of reopening old wounds,” Admiral Kirk in The Wrath of Khan.

One of the fundamental cruxes all artists bear is that their vision far exceeds their abilities to realize what they see in their mind’s eye.

Project #SQLImaginarium is one of them for me.

I had a grandiose plan and strategy, I lined up the skills, education, tools, and experience I would need to pull it off and…

Failure. Failure after failure. No one was paying me to do this, it was all spare-time efforts and frustration to learn and use the tools.

I flat-out dropped it at times. Then picked it back up and worked on it only to get upset and wash my hands of it. Rinse and repeat.

One positive that did come out of all of this was the idea and the creation of #SQLMemorial – one can see the latest version here.

If there ever is a fundamental flaw to me, is that my dreams, once set in motion, never die. They just keep coming back.

It has has been a long road for me. I’m not a game designer by trade or by degree or by formal training. I am completely self-taught.

Then VR showed up and I thought, “WOW!!!” I know a game engine now, just do a VR version of your vision!!!”

Easier said than done.

2018
https://toddkleinhans.wordpress.com/2018/05/09/and-so-it-begins-project-sqlimaginarium/

2019
https://toddkleinhans.wordpress.com/2019/05/14/project-sqlimaginarium-1-year-update/

2020
No update. Blame ‘Rona. 😦

2021
??? TBD

As time has marched on, my skills in these areas has dramatically improved over the years. Are my ideas too little, too late? Or was I simply ahead of my time?

I accepted a new job in the middle of this pandemic this year and I love it! I am also slowly coming out of my hiatus from several back-burner projects.

Personally I have accepted this new operational tempo of working from home under pandemic duress and am dealing with it. Some days are better than others, some days not so much.

2021 is going to see a lot of content coming from me and not all of it will be data related. So if a #tsql2sday topic doesn’t strike my fancy then I might not write about it. I’m into a lot of different things at this point of my life right now and not all of it deals with data. Data and the data community as much as I love it is NOT my life. I used to think that it was- data is me, it is my career, it is who I am, etc. But that is so short-sighted. Life is too short to define oneself in those terms. 2020 has seen the death of friends and family and acquaintances and I’m not going to choose to waste precious resources and time.

On a family trip long ago, I once visited Walden Pond with my Grandfather. Loved the book of the same by Henry David Thoreau and longed for simplicity in my life. For me 2021 is going to be a year of, “Simplicity, simplicity, simplicity!

Posted in Uncategorized | Leave a comment

PASS Has Passed And Is Now PASS

Within the last 72 hours as of this post (afternoon of 08Dec20), there have been three board members step down (read resigned) from the organization known as PASS.

This is heartbreaking to myself and others as PASS has helped not only my data career but the careers and jobs of many others. A membership of over ~300,000 is nothing to sneeze at- that is a significant number of professionals from around the world. There has been much speculation as to what is going to happen to it in the days ahead and I understand they have to follow appropriate legal and business protocols. I hope a white knight is going to save the day. If not and PASS dissolves or goes defunct or ceases to exist, a phoenix will rise from its ashes and people will connect, learn and share in other ways or in other organizations.

NOTE!!! I am not under any NDA, I’m not in the inner circle, I do not have any insider knowledge. Everything here are my own opinions, thoughts, and speculations.

I have been active in the local Denver SQL Server User Group for more than ten years. I’ve seen people and technologies come and go in that time, things are super hot and then deprecated and the like. Times change and people change with the times.

PASS used to stand for Professional Association of SQL Server then it just became a brand name with no real meaning behind the acronym.

A while back I tossed around the idea of a re-branding of sorts:

The Professional Association of SQL Solutions. Still PASS from a brand name, URL, trademark, etc. but a focus on data platforms which run queries. Required membership dues like other professional organizations. Beholden to no one other than the members and any future sponsorships are treated as grants, aka free money- no strings attached. When in-person events start back up, then look at tiered offerings. Consistent user group/event platforms like Bevy and Zoom for speakers (paid/compensated for their presentations too).

This would mean becoming inclusive of ALL data platforms, not just the ones Microsoft builds or supports. Does this mean including that OTHER platform, SQL Servers’ arch-enemy and the endless butt of many jokes over the years???

Yes. It would include Oracle and things Oracle has taken over, like MySQL.

“Are you a data professional who uses any type of query in your full-time job? We would like you to join us.”

With the rise of the cloud, a pay-per-query model is now here and there will always be a market for people who can write and tune queries. Until we are replaced by robots- wasn’t that supposed to happen any day now? 🙂

To paraphrase a quote from one of my presentations, The Life Cycle of a Query in VR (in my best, late, great, Carl Sagan voice), “Billions and billions of queries are run everyday. Knowing the cycle of how EACH one runs and what it touches to manipulate data is to better understand the data universe around us.”

For those of you who do not know, I maintain #SQLMemorial which can be viewed here. Never in a million years would I have ever thought I would be putting the name of an organization that has been a big part of my data career on it. I hope it doesn’t come to that fate. Time will tell.

Posted in Uncategorized | Leave a comment

Harvest of Shame Meets Telerobotics

Introduction

In the course of researching food security years ago, I have been interested in using tech to address food, water, clothing, and shelter in a manner that is beneficial to the long-term for a group. But I quickly learned that there is more to truly helping people than just their physical needs. My mind takes me back to The Bible and the story of Joseph and the Famine.

I have always wondered, how did they store that much food? What did the storage look like to prevent spoilage, rot, rodents, etc.? Topical questions, I know.

It was in that context I first stumbled across a video called Harvest of Shame. I watch it every year and think about what has changed and what has not, now in the last 60 years. It first aired just after Thanksgiving in 1960. Note it is a period piece meaning it was a documentary at that time in 1960- it opens with a cigarette sponsor, smoking is prevalent, things one will see and hear cause me to personally be uncomfortable.

Original 1960 “Harvest of Shame” (52min): https://www.youtube.com/watch?v=yJTVF_dya7E

50 Year Update (5min): https://www.youtube.com/watch?v=rkV3oVn209s

Ernest Kennedy (not in the film but was a migrant worker during this period): https://www.youtube.com/watch?v=KWX5xLJjPnY

Immokalee: a story of slavery and freedom (25min): https://www.youtube.com/watch?v=zBc4cOg9-ks

NPR story about the end of the boycott in 2005: https://www.npr.org/templates/story/story.php?storyId=4706271

CBS: https://www.cbsnews.com/news/tomato-war-ends-at-taco-bell/

Dirty, dangerous and demeaning. Perfect job for a robot but alas, robots to replace farm workers are still expensive. Maybe someday?

Enter Stage Left: Telerobotics

The following is pure speculation by an amateur futurist 🙂 Robots and drones have been on my mind for quite some time and they are still ridiculously expensive to build, program and maintain but costs are coming down and capabilities are increasing.

Telerobotics

Note these robots have a human-in-the-loop; they are not autonomous.

Key technologies are going to make labor disruptions happen:

Original patents have expired on the Da Vinci Surgical System

They Did Surgery on a Grape” – (no sound)

VR-controlled robot

Starlink

Part of the Picture

Using the above, imagine commercial, industrial robots being picked and placed in crop fields using heavy-lift drones. My vision for what these might look like could be right out of science fiction and probably very creepy looking. Like giant spider creepy looking.

On the surface, using telerobotics controlled by operators located anywhere in the world sounds like expensive overkill to replace seasonal migrant farm workers. But this is more than just replacing humans with robots.

The Dark Side

I’ll come right out and say it- I have concerns about the potential for high-tech being used to exploit labor costs. Skilled and unskilled labor markets are going to be potentially affected. Imagine if you will super low-cost labor with access to the above technologies. Fruit and vegetables in America being harvested by operators on the other side of the globe in a developing country. Foreign workers who never have to travel to a host country. Incarcerated, Children, Elderly (ICE) labor at below current piecemeal rates. Widgets getting assembled by a multi-national labor force switching between operators from beginning to end depending on the cost of vetted spot-labor.

The Bright Side

Robots are perfect for the 3Ds: dirty, dangerous and demeaning. The intersection of the technologies above will create new jobs for remote operators, remote console manufacturing, industrial robots, programming, drones, service and maintenance of it all- I think it will be neat to watch how all of this will transpire.

Conclusion

As I count my blessings this Thanksgiving, I remember where food comes from- many people working hard under harsh conditions oftentimes for back-breaking hours on end. Technologies can be used to allieviate some of that suffering but how to do that without also introducing new levels of suffering and problems.

Posted in Uncategorized | Leave a comment

T-SQL Tuesday #132: How Are You Coping with the Pandemic? Power Naps For The Win (FTW)!

This month’s T-SQL Tuesday is being hosted by Taiob Ali (T) on the topic of ideas or routines for coping with the pandemic. Thanks Taiob!

Sleeping on the Job

Call me old school. Back in the day, if one was caught sleeping on the job, that was a fireable offense and one could lose their job. The concept of even taking a quick nap was verboten.

For years I thought well, what if I just go out to my car and catch a few winks there? It can work and I have done that in the past but it is not comfortable plus I’m kind of self-conscious about it- I don’t want a passerby to think I’ve suffered some kind of accident and then notify the authorities and cause an issue. Even to avoid co-workers wondering about me I have been known to drive someplace just so I could nap out of sight…

Sleeping on the Job – New and Improved!

A skill or natural ability I have is to be able to fall asleep quickly. My wife hates it as it takes a while for her to wind down in the evenings. Me? I can lay down and be sleeping in roughly 10 minutes. This has caused her to wake me back up if we were in the middle of a conversation and she wanted to finish talking about something.

I have been mostly working from home for the last two+ years. When the pandemic hit, many people were forced to work from home (WFH). For me my day to day really did not change. However I was affected by the pandemic too as we all were and it disrupted my life.

While I am not an expert on the art of the Power Nap, I am aware and highly endorse all of its benefits. I try to stick to a tight schedule but alas in today’s crazy meeting driven WFH lifestyle I cannot always take a power nap when I want to. A siesta is always the easiest way for me to take a power nap right after lunch whenever that occurs.

I’ll set an alarm on my smartphone (usually 45 minutes from when I start), grab a warm blanket, and find a comfortable surface (sofa or bed but usually not under the covers), and focus on falling asleep. I know for many of us, working in IT doing something like programming it can be hard to un-plug. We lose our head of steam and it can take a while to get back up to speed and this can be a source of keeping one awake when one wants to fall asleep!

From the start of the power nap time (wind-down, actual sleep, wake up) I can maybe squeeze in half and hour of sleep. Do I take more than one power nap a day? Sometimes, especially between meetings. This is the challenge- how does one temporarily silence their smartphone and other distractions to sleep while still maintaining the availability to be reached in the case of an emergency? Block the time on your calendar as personal time- YES!!! With our WFH schedules and lifestyles this shouldn’t be a problem- even talk to your boss about it. As long as one is putting in the work hours, I think most places would be accommodating. Or don’t put it on your schedule and try to nap between “interruptions” haha.

A Memory From My Yoot

I use that word a lot. At the risk of mansplaining a yoot, here it is. When I was in high school, I knew this one guy who was a crazy smart programmer who carried a briefcase to school. Heh, remember briefcases? Odd thing was, we never saw him open it. What was in it we wondered? Programming secrets? Stealing things from the school? Why does a high school kid need a briefcase, what is he hiding???

Then one day, I was passing by the library during lunch and I saw it! He was in the back of the library with his briefcase open.

I couldn’t believe it, the mystery was solved!

The briefcase contained a giant pillow and his head was down, taking a power nap.

To quote the late great Paul Harvey once again, “…and now you know, the rest of the story.”

Posted in Uncategorized | Leave a comment

T-SQL Tuesday #131: Bean Counters and Databases

This month’s topic is being hosted by @sql_r – thanks Rob!

“For this month’s T-SQL Tuesday, I’d like you to write about your favorite analogies that help explain database concepts to people who aren’t database experts. I like the spirit of Reddit’s Explain Like I’m Five (ELI5) subreddit, where the explanations are meant for someone who has no previous assumptions about the topic.”

At a dinner party long ago, I was asked what I did for a living and was met with some odd looks. Truth be told, numbers are not my thing and I barely passed my accounting and finance classes in college.

Trip Down Memory Lane

One of my favorite documentaries about computer history is called Accidental Empires by Robert X. Cringley (TV version was called Triumph of the Nerds). There was a scene where before the electronic spreadsheet was invented and coded into software (VisiCalc) a professor was standing in front of a chalkboard. The kind that created nasty chalk dust and make a distinct sound when writing with it. This professor was writing a pattern of grids on the board – it was a chalk version of a spreadsheet. When he needed to re-calculate a set of cells, he make the calculation on a TI Business Analyst hand calculator, erased the previous results, then wrote in the new calculations.

Dan Brinklin got the idea that this could be done on the computer via software and the professor started shaking with anticipation, “You don’t understand, this is what I have to do all day!!!” Visions of helping accountants and bookkeepers and business people really spurred him on.

Back To The Party

So I struck up a conversation with someone who was an accountant (also called a bean counter, outside of financial circles). They asked what I did for a living. I said I worked with databases and tried to expand on that…

“A database is set of two computer files, a data file and a log file. Got to be sure they stay healthy!”

More blanks stares. I was going to try something different. Hold my beer.

“A data file is like a chart of accounts. It has everything you need to run the business. The log file is like the general ledger– transactions get posted from here into the chart of accounts. Between the two, the general ledger is the MOST important as it contains a sequential record of all of the transactions. As a database administrator, keeping the log file of any database backed up and running smooth is very important to me.”

A light bulb went off in their head. They understood what I was saying in a language they could understand.

Conclusion

Try to put things into terms your audience can understand, especially those outside of your technical expertise.

Posted in Uncategorized | Leave a comment

T-SQL Tuesday #130 – Automation Using Python

This month’s #TSQL2sday topic is being hosted by Elizabeth Noble, known on Twitter as @SQLZelda

Thanks Elizabeth!

From her post, I picked: “…If you haven’t had a chance to automate some part of your job, what would you like to automate and what are your hurdles?”

My Passion Outside of Work – Virtual Reality (VR)

So I’m into VR. You might have seen SQLMemorial (although I’ve updated the workflows since that post and, sadly, added some more people- latest video is here). I love building and doing things in VR. I use Epic Games Unreal Engine (UE4) and a host of other tools.

Automate the Boring Stuff With Python

I’ve mentioned this book before (ATBSWP) and it’s on-line content. It has been updated to the second edition and focuses exclusively on Python 3. Yay! It’s a great way to both learn Python and do useful things with it. Lots to learn and try.

Scripting the Unreal Engine Editor Using Python

While you cannot use Python when a game is running using UE4, one can however script out A LOT of things within the editor itself. Although until the VFX Reference Platform is completely updated to Python 3, sometime in 2020, UE4 natively runs 2.7 – boo. But! One can make some changes and re-build the ‘Engine from C++ source (~45 minutes on my desktop rig). Re-building from source is not for the faint of heart but it is possible- I figured it out!

The documentation is a great place to start. Manually manipulating objects is a slow and error-prone process and being able to automate common tasks within the Editor is a huge time-saver. Plus there are a ton of free scripts built by the community on github and other places.

Scripting Azure Data Studio or SSMS with Python

This is the benefit or curse of having multiple technical interests- it makes one ask if something works here, then why can’t it work there?  Food for thought.

Why Can’t We Easily Run Python in SSIS?

At work I am investigating using SSIS to connect to various data sources and do ETL tasks. I would love to be able to use Python. Sadly one can only execute Python scripts as an Execute Process Task, not as a Script Task. Also, one doesn’t have any control over Execute Process Task – either it works or it doesn’t and one has to have different logging set-up to troubleshoot errors and monitoring. Script Task can do VB and C# but not Python. I don’t know why that hasn’t been implemented in SSIS. I also know one can run T-SQL Python scripts but honestly that syntax is still kludgy to me and one loses Visual Studio Intellisense, step in, step out, breakpoints, unit testing like one can do with .NET languages. We are slowly moving things into the cloud so maybe I just need to figure out the hybrid or pure cloud approach.

And Now For Something Completely Different – Desktop Supercomputing

Too many projects I want to pursue. I’ve always been fascinated by the concept of cluster computing and now with Raspberry Pi, one can explore building and using them at a modest cost. Raspberry Pi Supercomputing and Scientific Programming talks about building them and if you want to buy a fully built system, take a look at picocluster. And what language can one program them with? Python, of course!

Conclusion

Python is a great language to learn and do things with- the more one uses it on a day to day basis, the easier it becomes to think about doing more and more things with it. Now with Python 3 here to stay, I’m slowly on my way to becoming a better Python programmer to help ATBSWP and all of the fun stuff too.

Posted in Uncategorized | 1 Comment

T-SQL Tuesday #129 – Diamonds and Rust \m/-_-\m/

This month’s T-SQL Tuesday “Let’s Build a Time Capsule” topic was created and is being hosted by Tamera Clark. Thanks Tamera!

Nostalgia

We have a fondness to remember things from the past with nostalgia. This is a normal human emotion to feel. I have a “thing” for 80’s heavy metal as I grew up during that time and went to several concerts. Remember going to concerts??? So when this SQL time capsule gets opened up sometime in the future, it will be remembered with generally positive thoughts and feelings. It is also supposed to be a reflection of things at the time it was sealed up- the good, the bad, and the ugly.

Diamonds and Rust

I love the cover version by Judas Priest of Joan Baez’s song Diamonds and Rust. And in memory of the late Peter Green The Green Manalishi (With the Two Prong Crown) from the same tour.

Memories can bring back a whole host of thoughts and feelings. Sometimes we even recall things incorrectly, as in things did not happen exactly as we once thought they did. Hence the title of my post in reference to the song of the same. Rusty things which were once beautiful and diamonds with a patina of rust on them (although diamonds really don’t rust, heh).

Data Archival

As data professionals, we wring our hands about consuming space from backups. I have done a lot of research in this area and one of the more interesting technologies I stumbled across is the Rosetta Disk. Imagine instead of storing data for archival using magnetic fields, the ones and zeroes are etched into a surface which can last for 10,000 years. Best part is that it only requires an optical device like a microscope to read- no electronics are required.

From the #SQLFamily what would I want to be etched on it? A copy of the latest SQLMemorial. A dump from Twitter of #SQLhspics of course. And maybe a PDF of Microsoft SQL documentation. And maybe a copy of SQL Server 2000 Developer Edition too just for grins and giggles.

Posted in Uncategorized | Leave a comment