This no more is the thing with only IT companies, the Mechanical companies, The electrical companies, The Manufacturing Industry, The Hospitals, The Banks, The Telecom Companies all are using the Data Warehousing, since all have understood the power of storing the data in a sophisticated manner and using it for Reporting and Analytical purposes.
The DATA warehousing, as the name suggests is a game played with data. The data for today, yesterday, the day before yesterday, a week old data, a year old data stretching may be up to the ages of your grandfather.
The data warehouse is mainly the term used in the companies to manage the data. The larger the company, the bigger the data, the bigger the requirement to manage the data. The concept of a data warehouse is to have a centralized database that is used to capture information from different parts of the business process. The definition of a data warehouse can be determined by the collection of data and how it is used by the company and individuals that it supports. Data warehousing is the method used by businesses where they can create and maintain information for a company-wide view of the data.
The History and Evolution of Data Warehousing
Many people feel that data warehousing is only copying data from one place to another, They feel “That doesn’t make any sense! Why waste time copying and moving data, and storing it in a different database? Why not just get it directly from its original location when someone needs it?” This is totally incorrect. Data Warehouses are very much evolved as compared to Databases.
Data Warehouses do far more than just storing the data.
Going to 1970-80's years back, the computing world was dominated by the mainframe in those days. Real data-processing applications, the ones run on the corporate mainframe, almost always had a complicated set of files or early-generation databases (not the table-oriented relational databases most applications use today) in which they stored data.
Although the applications did a fairly good job of performing routine data processing functions, data created as a result of these functions (such as information about customers, the products they ordered, and how much money they spent) was locked away in the depths of the files and databases. The way data was stored made it more complex to solve the purpose. It was almost impossible, for example, to see how retail stores in one region were doing against stores in the other region, against their competitors, or even against their own performance in some earlier period.
Between 1976 and 1979, the concept grew out of research, driven from discussions with Citibank’s advanced technology group, called TERADATA. The name Teradata was chosen to symbolize the ability to manage terabytes (trillions of bytes) of data.
Early days of Data Warehousing - 1980's
As the era began, the computers were the only name making there presence everywhere. The organization started to have computers everywhere in each department. How could an organization hope to compete if its data was scattered all over the place on different computer systems that weren’t even all under the control of the centralized data processing department? (Never mind that even when the data was all stored on mainframes, it was still isolated in different files and databases, so it was just as inaccessible).
Special software that came into existence which made the life simple for the user and everyone, using the data or analyzing the data. This new type of software, called a distributed database management system (distributed DBMS, or DDBMS), would magically pull the requested data from databases across the organization, bring all the data back to the same place, and then consolidate it, sort it, and do whatever else was necessary to answer the user’s question.
Although this was a thing all were expecting, but the chocolate is always better if its sweeter, the sweetest the perfect. The chocolate being RDBMS (relational database management system) for decision support — the world’s first.
Why Data Warehouse over Normal Databases !!!
The need has been established for a company-wide view of data in operational systems. Data warehouses are designed to help manage and businesses analyze data and this helps to fill the need for subject-oriented concepts. Integration is closely related to subject orientation. Data warehouses must put data from disparate sources into a consistent format. They must resolve such problems as naming conflicts and inconsistencies among units of measure. When this concept is achieved, the data warehouse is considered to be integrated.
The form of the stored data has nothing to do with whether something is a data warehouse. A data warehouse can be normalized or denormalized. It can be a relational database, multidimensional database, flat file, hierarchical database, object database, etc. Data warehouse data often gets changed. Also, data warehouses will most often be directed to a specific action or entity.
Data warehouse success cannot be guaranteed for each project. The techniques involved can become quite complicated and erroneous data can also cause errors and failure. When management support is strong, resources committed for business values, and an enterprise vision is established, the end results may turn out to be more helpful for the organization or business. The main factors that create needs for data warehousing for most businesses today are requirements for the company wide view of quality information and departments separating informational from operational systems for improved performance for managing data.
With this basic history of Data Warehouse, let's try to gather more information about the Giant - The Data Warehouse. There are numerous pages, blogs, websites these days which talk about Data Warehousing. No doubt a few of them have really good information.
I have gone through many of them – It's great to see the enormous knowledge people gather in there lifetime and the better work they do is share that knowledge with others.
When I shuffle through the websites, blogs I sometimes feel annoyed that none location provides everything at its best. Definition of Data Warehousing is written in a great manner in one blog, OLAP and OLTP are good at other ........!!!
Other bad things – Who the hell is going to tell me how do I relate all these to the world around me so that I can understand its importance.
Now try reading a few other pages on the internet – and you will understand what I am trying to say in my last sentence above.
After working of quite a few Data Warehousing projects and gathering lots of information from the pioneer's article - here are few of my real life examples for all the topics which we say “DATA WAREHOUSING CONCEPTS” !!!
I will try to provide some real-life scenario for each concept which will be useful in helping you gather real-life knowledge of Data warehousing.
Learn Well Technocraft Provides the Best Informatica, Abinitio, Tableau, PowerBI, Data Science, Machine Learning, Artificial Intelligence, and Python Analytics, etc.