Reform Your Business Intelligence with Lean, High-Performance Solutions
Conceived in 1918, my elderly neighbor grew up amid the Depression period and cherished the autonomy that working and driving offered for the duration of her life, even though she never did figure out how to top off her gas tank. One would not like to take freedom too far when it’s all said and done. In addition, she didn’t need to — her late spouse, a WWII armed forces veteran, constantly demanded that it was his obligation to do it for her. When he passed away, her collaborators at the plant where she worked assumed control over this errand, followed by her supervisor who proceeded with the custom after she resigned after 30 years.
This story shows a bigger point about grasping change — while a determined lack of awareness can have its advantages in a cultured society, an official initiative that keeps up stiff-necked obliviousness of their own information management and business intelligence operations can result in genuine harm to their venture’s bottom line. With a breakthrough in hybrid technologies that employ both data warehouse appliances and Hadoop, companies can now better serve their own Business Intelligence engine in today’s challenging economic climate.
Start by exciting the troops. As far as enhancing ‘Big Data’, EDW, and Business Intelligence Solutions, numerous organizations still work under a bygone, “let the IT fellows/programming supplier/specialty unit manage it” attitude. Notwithstanding, I’ve discovered that the best organizations can top off their information gas tank and power it productively. How? Rising advances in leaner, hybrid BI strategies and architectures are right now picking up a considerable measure of momentum, effectively moving customers towards data management and BI solutions.
This technology is similar to the hybrid vehicle concept that endeavors to wean our population off of our dependence on fossil fuels. Hybrid cars are a pragmatic, cost-effective approach automakers have adopted in response to 1) increased consumer demand for more practical, fuel-efficient vehicles; and 2) newer, stricter government regulations aimed at countering documented increases in greenhouse gas emissions.
An enterprise’s “3 Vs” of data — volume, velocity, and variety — are what determines the types of technology that are warranted. If my neighbor was still driving today, I wonder if she would ever consider an electric car with a charger in the garage. Why? Because her “3V” threshold has always been pretty low; all of her driving was short, local trips within ten miles of home.
Similar to my neighbor, Companies need to evaluate their 3 Vs and accurately determine their threshold to embrace new approaches. Calling these approaches “lean” solutions, they take advantage of emerging cutting-edge tools, technologies, and processes to give organizations much quicker access and control of their data, in real-time, streamlining processes and reducing the overall cost of ownership.
Data warehouse technology is ideal for organizations that have 3-V thresholds that are also under certain data parameters and guidelines. A “lean” Data warehouse approach involves the acquisition of all available data, not just slices of data, and applies principles of real-time computing to allow permutations of that source data, using virtual source-of-record images of the data, not the physical data itself.
The amount of required data transformation layers is reduced from over 10, in many cases, to three steps — Source Of Record, Insight, and Consumer — greatly decreasing complexity and overall processing times. By employing these lean Business Intelligence techniques, Data warehouses offer performance advantages and cost savings that can’t be denied.
Rechargeable electric cars offer performance advantages that include high efficiency low emissions and less noise, although the sticker price may be a bit higher and travel distances have a limited range. Likewise for new Business Intelligence consumers, a scalable data warehouse appliance can really go a long way in getting your business over those first 3-V hurdles.
Using lean concepts with Data Warehouse technology, rapid changes are straightforward, overall project costs are lower, due to simpler design and less need for data modelers and developers, and maintenance is much less complicated. Lean DWA approaches also offer increased functionality since all data is available. Business units can consume information on their terms, when they’re ready, without having to depend on other business units.
Can an enterprise’s 3-V capacity outgrow a Data Warehouse solution? Yes and no. As a rapid-deployment solution, DWAs are also highly flexible, designed to adapt to an organization’s growth and increased need for the 3V components of big data.
However, there is a natural inflection point in the life of a DWA-based lean solution, when significantly more volumes and variety of data will be needed at faster speeds. The best new Business Intelligence solutions embrace the concept of hybrid technology. Hybrid Business Intelligence solutions combine new versions of traditional Business Intelligence architectures with new technology like DWAs.
Gain the most benefits from a hybrid lean Business Intelligence solution. What if my neighbor added more, longer destinations to her routine, for example, a visit to her sister who lived 25 miles away? When this distance proved too much for her, an add-on solution could be called for in the form of a taxi, limo, or an accommodating friend. Problem solved.
For those of us who travel longer distances as a rule rather than the exception, however, the better bet would be a hybrid car which combines a new, high-efficiency gas engine with a rechargeable, electric battery component. High-efficiency, gas-powered engines — which leverage older internal combustion technology in new, more efficient ways — are still better suited for heavy, long-distance driving and hauling, while electric power offers better performance and efficiency on shorter distances.
Right now, the best Business Intelligence solutions successfully balance the benefits that proven traditional processes offer with newer technologies that foster self-reliance and don’t drain valuable resources, energy or time. Hybrid Business Intelligence solutions employ “electric car” Data Warehouse technology to take advantage of its more nimble and single-source approach for lower 3 V processes.
To accommodate big data applications that have much heavier processing demands that a Data Warehouse couldn’t handle or become too costly to scale, hybrid Business Intelligence solutions add one or more technologies (like Hadoop), which can tackle more complex processes on less expensive commodity hardware. Hadoop, like new highly fuel-efficient engines that that are based on yesterday’s internal combustion technology, is much more cost-effective than a DWA to pre-process semi- or unstructured data and store massive amounts of big data.
The ultimate hybrid lean solution would give an enterprise the ability to drive and maneuver its 3 V threshold through all kinds of terrain at top speeds while conserving energy at the same time. Can it be done? Absolutely, with the keys being cost, processing requirements, and the 3 V threshold. Let’s consider the cost factor first. As more and more data is brought into your enterprise data warehouse (whether it is a Data Warehouse or traditional database), the footprint of the database increases, which drives costs up and impacts performance.
Next, let’s factor in the processing requirements. Not all processes need to be executed within your EDW. Storing unused data in an EDW is just a bad, wasteful practice — sure, it’s easier, but much more costly. An enterprise should complete an audit of all its processes to determine if it makes more sense to bring certain data and processes into its EDW, or execute it within in a pre-storage and pre-process Hadoop environment.
Of the 3 V’s, variety typically means semi- or unstructured data, which a database neither stores nor processes very well. As the variety of data increases, the demand to take the unstructured data and structure it in a pre-process becomes greater. This pre-process is another good example of how to leverage a big data Hadoop environment. The other two V’s — volume and velocity — are all about scale and processing power.
Again, the tipping point should focus on the following test: “Should I process and store this volume and velocity within the EDW environment, or better yet, pre-process and pre-filter this data, and only bring in the data that is required — already analyzed, compared and evaluated — into the Data Warehouse ?”
A truly balanced hybrid lean framework optimizes the total cost of ownership by maintaining speed, functionality, and flexibility. Adding additional architecture allows for exponential increases in processing power while significantly increasing amounts and types of available source data. If your business finds it would gain more measurable and meaningful intelligence from integrating these larger stores of unstructured, semi-structured, and structured data, hybrid technology provides much-needed access to proven ETL tools and Hadoop technology, greatly extending the life of your EDW technology investment.
Widowed in her 40s, my neighbor always bought American-made cars to honor her late husband’s military service. He swore by Pontiacs and Buicks, and that was always good enough for her. While Pontiac is no more, Buick’s website describes its 2014 Lacrosse performance and fuel efficiency this way: “Who says luxury and performance can’t coexist? Opt for the 304hp V6 engine and you’ll be enjoying the road in no time. Innovations like eAssist technology help conserve fuel (EPA estimated 25 mpg city/36 mpg hwy.” Similar to how U.S. car manufacturers like Buick are making much better use of the earth’s energy resources with hybrid vehicle innovations, businesses that work to understand hybrid technology’s long-term advantages and deploy appropriate hybrid solutions will gain much better governance, internal control, and overall use out of their big data assets. Hybrid technology affords enterprises a better opportunity to grow and prosper in what seems a perennial challenging economy.
Are there better solutions in the future? Of course. Innovators from all walks of life, not just automakers and data scientists, are always driven to seek what’s beyond the next hill when it comes to developing better-use strategies for dwindling resources while not harming. In the meantime, incorporating hybrid Business Intelligence technology can satisfy your company’s need for better business intelligence — for the foreseeable future — with more user-friendly, agile frameworks while further advancements in Business Intelligence solutions are developed and refined. Don’t forget, though, to make sure you have a full tank of gas and your battery is charged — and that you know how to do these tasks yourself if you have to.
ExistBI has distinguished itself as one of the top Data Warehouse Consulting Companies in the world. To discuss your business intelligence/enterprise data warehouse requirements simply give our Data Warehousing Experts a call today: United States / Canada +1 (310) 229 5775 or UK / Europe +44 (0)207 554 8568 or fill in the contact form on their website at: https://www.existbi.com/contact-us.
