<\/span><\/h2>\n\n\n\nGenerally information in companies is found in work tools such as a CRM, ERP, accounting software, management software or even Excel, which are perfectly designed to insert data, but are not created to show information in real time, for example. <\/p>\n\n\n\n
Their databasess are optimal for data insertion but not for data extraction.<\/p>\n\n\n\n
This is why one of the first jobs to be done when a company decides to take the leap and exploit its data to the fullest is to locate all these sources of information and extract the data to be dumped into a suitable database for their transformation, extraction and visualisation<\/strong>. <\/p>\n\n\n\nThis type of database is known as a Data Warehouse. <\/p>\n\n\n\n
<\/span>Diferences between Data Warehouse, Data Lake and Data Mart<\/span><\/h2>\n\n\n\nThe biggest challenge in all BI projects is the quality of the source data, its location, extraction and standardisation. <\/p>\n\n\n\n
This is a simple process map of a typical BI project. <\/p>\n\n\n\nTraditionally, the procedure for exploitation involves inserting the raw data, “in raw form”, into a database, which is know as Data lake<\/strong>. Data lakes are not organised or structured<\/strong>, they are simply, as their name suggests, a data lake. <\/p>\n\n\n\nIt is in this lake that information is “fished” to be processed and organised in a Data Warehouse. The data in the Data Warehouse have been previously worked on through programming processes known as ETL<\/strong>.<\/p>\n\n\n\nOnce the data is transformed and organised it is loaded into another database, or several, smaller ones than the data warehouse, which are known as Data Mart(s). Each Data Mart usually contains the data of a specific area of the company: finance, sales, marketing… <\/p>\n\n\n\n
It is necessary to have these three types of databases? No. <\/strong><\/p>\n\n\n\nIt will depend on the BI project that needs to be carried out, the point in which the company is located, the volume of data, the loading speed required, the immediacy of the data required…<\/p>\n\n\n\n
<\/span>ETL Processing<\/span><\/h2>\n\n\n\nExtract, Transform, Load<\/strong>. These are lines of programming code that allow the combination of the data we have extracted from different sources and that may have different formats, so they can work together. ETL porjects can be written in multiple programming languages, such as Java, Python, SQL…<\/p>\n\n\n\n<\/span>Which technology shoud I use to process my company’s data?<\/span><\/h2>\n\n\n\nTo make this decision the most important thing is to have the help of a proffesional <\/strong>who knows all the possibilities. Today, Big Data techcnology is very extensive and it is essential to know which tools to use. In order to not to go into technical detail, we will summarise the main points in this section. <\/p>\n\n\n\nThe first thing you shoud know is that the BI project<\/strong> you carry out can work in traditional hosting platform (contracting physical servers to host the project) or in the Cloud, whose main characteristic is that the information is replicated between different network nodes and not in a single physical server. <\/p>\n\n\n\n<\/span>The advantages of using cloud<\/span><\/h2>\n\n\n\nFor a Big Data project that, by its very nature, will tend to grow steadily, Cloud services are usually the most appropriate as they are flexible and increase according to the project needs. To make it simple, they work like electricity at home: you hire the Cloud service and you pay depending on the consumption (storage, consultation, speed…).<\/p>\n\n\n\n
this feature is one of the reasons why it is essential to have experts when using cloud services such as Amazon, Azure (Microsoft) or Google Cloud, as costs can shoot up if resources are not correctly optimised. <\/p>\n\n\n\n
If well used, the advantages of the Cloud are obvious. Lower cost and greater profitability, greater consistency (if one node fails, the information is automatically replicated to another), greater security or availability of multiple features on each of the available platforms. <\/p>\n\n\n\n
<\/span>How do I choose the database I need?<\/span><\/h2>\n\n\n\nAnother important point about technology in BI projects is the type of database to be used, or the tools used to carry out the ETLs<\/strong>. <\/p>\n\n\n\nWith respect to databases, the most obvious difference is the amount of data to be processed and the speed required for its representation. Here again, having professionals who know and control all the option is the key. <\/p>\n\n\n\n
Relational databases are those that have been used in software and business applications over the last 40 years. They can be used to process Big Data, but they may well fall showt of what is requiered. Precisely because of the need to easily handle huge amounts of data<\/strong>, other types of databases have emerged that do not use tables, fields and rows like relational ones and that are know as NoSQL.<\/strong> <\/p>\n\n\n\nJust to name a few, the most commonly used databases for BI projects are Orable, MongoDB, Amazon’s Redfshit or Kassandra, among others.<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"
The need to bring order into the chaos of information<\/p>\n","protected":false},"author":1,"featured_media":1205,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[63],"tags":[],"theme":[61],"class_list":["post-1082","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-talk-en","theme-smart-visual-data"],"acf":[],"yoast_head":"\n
Master the data - Smart Visual Data<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Master the data - Smart Visual Data\" \/>\n<meta property=\"og:description\" content=\"The need to bring order into the chaos of information\" \/>\n<meta property=\"og:url\" content=\"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/\" \/>\n<meta property=\"og:site_name\" content=\"Smart Visual Data\" \/>\n<meta property=\"article:published_time\" content=\"2021-01-04T10:10:43+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-08-07T12:15:52+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/pre.zeus.vision\/wp-content\/uploads\/2021\/01\/data_visualization_tools_smart_visual_data-3.png\" \/>\n\t<meta property=\"og:image:width\" content=\"455\" \/>\n\t<meta property=\"og:image:height\" content=\"521\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#\\\/schema\\\/person\\\/4087f787623e5a4f7c5222093fd55822\"},\"headline\":\"Master the data\",\"datePublished\":\"2021-01-04T10:10:43+00:00\",\"dateModified\":\"2023-08-07T12:15:52+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/\"},\"wordCount\":974,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/zeus.vision\\\/wp-content\\\/uploads\\\/2021\\\/01\\\/data_visualization_tools_smart_visual_data-3.png\",\"articleSection\":[\"Data Talk\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/\",\"url\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/\",\"name\":\"Master the data - Smart Visual Data\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/zeus.vision\\\/wp-content\\\/uploads\\\/2021\\\/01\\\/data_visualization_tools_smart_visual_data-3.png\",\"datePublished\":\"2021-01-04T10:10:43+00:00\",\"dateModified\":\"2023-08-07T12:15:52+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#primaryimage\",\"url\":\"https:\\\/\\\/zeus.vision\\\/wp-content\\\/uploads\\\/2021\\\/01\\\/data_visualization_tools_smart_visual_data-3.png\",\"contentUrl\":\"https:\\\/\\\/zeus.vision\\\/wp-content\\\/uploads\\\/2021\\\/01\\\/data_visualization_tools_smart_visual_data-3.png\",\"width\":455,\"height\":521,\"caption\":\"Master the data with data visualization tools\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/pre.zeus.vision\\\/en\\\/data-talk-en\\\/master-the-data\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Portada\",\"item\":\"https:\\\/\\\/zeus.vision\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Master the data\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/zeus.vision\\\/en\\\/\",\"name\":\"Smart Visual Data\",\"description\":\"Turn data into business\",\"publisher\":{\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/zeus.vision\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#organization\",\"name\":\"Smart Visual Data\",\"url\":\"https:\\\/\\\/zeus.vision\\\/en\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/zeus.vision\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/logo-zeus.svg\",\"contentUrl\":\"https:\\\/\\\/zeus.vision\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/logo-zeus.svg\",\"caption\":\"Smart Visual Data\"},\"image\":{\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/zeus.vision\\\/en\\\/#\\\/schema\\\/person\\\/4087f787623e5a4f7c5222093fd55822\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/37134783f2af5bc6d1808b28599e8ecc4edc5932648e0b335dfcff0362679e7b?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/37134783f2af5bc6d1808b28599e8ecc4edc5932648e0b335dfcff0362679e7b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/37134783f2af5bc6d1808b28599e8ecc4edc5932648e0b335dfcff0362679e7b?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"sameAs\":[\"https:\\\/\\\/zeus20.zeus.vision\"],\"url\":\"https:\\\/\\\/zeus.vision\\\/en\\\/author\\\/admin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Master the data - Smart Visual Data","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/","og_locale":"en_US","og_type":"article","og_title":"Master the data - Smart Visual Data","og_description":"The need to bring order into the chaos of information","og_url":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/","og_site_name":"Smart Visual Data","article_published_time":"2021-01-04T10:10:43+00:00","article_modified_time":"2023-08-07T12:15:52+00:00","og_image":[{"width":455,"height":521,"url":"https:\/\/pre.zeus.vision\/wp-content\/uploads\/2021\/01\/data_visualization_tools_smart_visual_data-3.png","type":"image\/png"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#article","isPartOf":{"@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/"},"author":{"name":"admin","@id":"https:\/\/zeus.vision\/en\/#\/schema\/person\/4087f787623e5a4f7c5222093fd55822"},"headline":"Master the data","datePublished":"2021-01-04T10:10:43+00:00","dateModified":"2023-08-07T12:15:52+00:00","mainEntityOfPage":{"@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/"},"wordCount":974,"commentCount":0,"publisher":{"@id":"https:\/\/zeus.vision\/en\/#organization"},"image":{"@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#primaryimage"},"thumbnailUrl":"https:\/\/zeus.vision\/wp-content\/uploads\/2021\/01\/data_visualization_tools_smart_visual_data-3.png","articleSection":["Data Talk"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/","url":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/","name":"Master the data - Smart Visual Data","isPartOf":{"@id":"https:\/\/zeus.vision\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#primaryimage"},"image":{"@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#primaryimage"},"thumbnailUrl":"https:\/\/zeus.vision\/wp-content\/uploads\/2021\/01\/data_visualization_tools_smart_visual_data-3.png","datePublished":"2021-01-04T10:10:43+00:00","dateModified":"2023-08-07T12:15:52+00:00","breadcrumb":{"@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#primaryimage","url":"https:\/\/zeus.vision\/wp-content\/uploads\/2021\/01\/data_visualization_tools_smart_visual_data-3.png","contentUrl":"https:\/\/zeus.vision\/wp-content\/uploads\/2021\/01\/data_visualization_tools_smart_visual_data-3.png","width":455,"height":521,"caption":"Master the data with data visualization tools"},{"@type":"BreadcrumbList","@id":"https:\/\/pre.zeus.vision\/en\/data-talk-en\/master-the-data\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Portada","item":"https:\/\/zeus.vision\/en\/"},{"@type":"ListItem","position":2,"name":"Master the data"}]},{"@type":"WebSite","@id":"https:\/\/zeus.vision\/en\/#website","url":"https:\/\/zeus.vision\/en\/","name":"Smart Visual Data","description":"Turn data into business","publisher":{"@id":"https:\/\/zeus.vision\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/zeus.vision\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/zeus.vision\/en\/#organization","name":"Smart Visual Data","url":"https:\/\/zeus.vision\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/zeus.vision\/en\/#\/schema\/logo\/image\/","url":"https:\/\/zeus.vision\/wp-content\/uploads\/2020\/07\/logo-zeus.svg","contentUrl":"https:\/\/zeus.vision\/wp-content\/uploads\/2020\/07\/logo-zeus.svg","caption":"Smart Visual Data"},"image":{"@id":"https:\/\/zeus.vision\/en\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/zeus.vision\/en\/#\/schema\/person\/4087f787623e5a4f7c5222093fd55822","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/37134783f2af5bc6d1808b28599e8ecc4edc5932648e0b335dfcff0362679e7b?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/37134783f2af5bc6d1808b28599e8ecc4edc5932648e0b335dfcff0362679e7b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/37134783f2af5bc6d1808b28599e8ecc4edc5932648e0b335dfcff0362679e7b?s=96&d=mm&r=g","caption":"admin"},"sameAs":["https:\/\/zeus20.zeus.vision"],"url":"https:\/\/zeus.vision\/en\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/posts\/1082","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/comments?post=1082"}],"version-history":[{"count":4,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/posts\/1082\/revisions"}],"predecessor-version":[{"id":1112,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/posts\/1082\/revisions\/1112"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/media\/1205"}],"wp:attachment":[{"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/media?parent=1082"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/categories?post=1082"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/tags?post=1082"},{"taxonomy":"theme","embeddable":true,"href":"https:\/\/zeus.vision\/en\/wp-json\/wp\/v2\/theme?post=1082"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}