Extract/Transform/Load (ETL)

Published On - October 4, 2021

Makarand Patwardhan

ETL even though a very old concept, its popularity made it prominent in 1970.

The non-software fellows (like me) must be wondering what I am referring to, yes same was the case with me when I first time heard the term ETL.

I know I have taken lot of time but have not talked about what ETL stands for. In data warehousing ETL stands for Extract/Transform/Load.

Recently, while going through some of the literature on Data, I encountered this term after many years. Meanwhile as a regular practice I was cleaning the social media inputs, which is my weekly exercise. Some time you really get fed up with all good mornings, have a nice day, thought for the day messages. I was thinking how this ETL is relevant in today’s context of data overload in personal life. Because of advancement of technology on one side we are well connected, but I think, day by day we are getting emotionally disconnected. We are overloaded with data and some time it becomes too much.

Let’s understand the ETL concept as used in “IT” industry. It is a general procedure of copying data from one or more sources into a destination system which represents data differently from the source or in a different context than the source. Gotcha, you caught me and you are right, this is how it is defined in Wikipedia.

Going forward again from Wikipedia, properly designed ETL system extracts data from the source, enforces data quality and consistency standards, conforms data so that different sources can be used together, and finally delivers data in presentation ready format so that developer can build application and end user can make decision.

Some of you have already guessed where I am driving this point. In today’s personal life, the social media is throwing so much data at you, that person is fatigued at receiving, analysing (if he really does) and acting on that data. (Here I am not referring to “Forwarding”, but actually put to use). Now a days, I hear people saying abstaining from social media, somebody told me he is on vacation from social media, while the best term I came across was sabbatical from social media.

This means there is a mental fatigue because of overdose of information / data. During last one year as the WFH was/is in vogue, individuals are expressing / realising the value of human interaction in BC (Before Covid) era. Hence the social media are buzz with remedies, precautions, advises and so on and so forth.

So, if you are experiencing data fatigue, why not try ETL in social context.

Now if you apply ETL, then system Individual Reads data from the social media, enforces Personal data quality and consistency standards, conforms data so that different sources can be used together, and finally Absorbs / Rejects data so that Individual can build Wisdom andnSelf or to whom he forwards can make use of information wisely.

Another thought, can we apply “Blockchain” in social media? Confused? So am I, will write about this once I gather more insight.

If you look at your professional career as an average 35 years journey, you need to have both short term and long term focus. If we define the short term as initial 10 years of your career, it’s important to focus on learning operating stuff or functional skills as you begin your career. Most of the times/More often than not, your performance at this stage will be evaluated basis your speed of delivery, technical accuracy, enthusiasm, ability to work in a project team and above all, willingness to learn.
It’s important to build and nurture the core, especially when you have spent around 10 -15 years of your career (remember number of years are indicative, each one moves differently in their career journey based on their potential as well as opportunities). Building a strong core to me means developing your expertise, knowing well what you are really good at and also understanding what others think about your expertise. Here, you need to pick up new skills and expertise along with people and team management capabilities. At this stage you also decide which industry or sector you would like to be in and whether you want go deep and specialize in a field or widen your competency base and move into general management. Ideally, at this stage you should also experience what it means to work with different geographies, cultures etc., if you get such opportunities.
You grow rapidly in terms of knowledge, expertise, leadership capabilities, business acumen when you have spent around 20 years in your career. It’s not the number of years which defines this phase, but the kind of challenges you have handled, expertise you have built, teams you have developed or customers you have won, all of these help you to shape your future. You learn a lot from your achievement and equally from the mistakes you have made over the years. You build your presence, wherever you work and beyond the company as well, by taking keen interest in your profession and furthering it’s development. This phase also helps you to secure your future financially and make clear plans for the rest of your life.
Many a times you look beyond the obvious career trajectory and question what can be your next challenge or goal. Not that everyone thinks like that or needs to think in that way. Many people happily accept the “ceiling”, continue their journey and achieve their set career and life goals. While some take a leap and opt voluntarily for a new career challenge and opportunity to fulfill their vision. This kind of career transition is possible when you are truly attached to your core values and determined to walk the lane once again for a new quest and ambition and mentally strong to manage associated disruptions. You don’t measure the success here in defined societal standards, but the way you move closer to your vision and how much difference you are able to make to others; these become important parameters. Ultimately, who defines your success?
Share this post