Posts

Showing posts with the label Big data

SQL : PostgreSQL

  Aggregate Functions Like most other relational database products,  PostgreSQL  supports  aggregate functions . An aggregate function computes a single result from multiple input rows. For example, there are aggregates to compute the  count ,  sum ,  avg  (average),  max  (maximum) and  min  (minimum) over a set of rows. As an example, we can find the highest low-temperature reading anywhere with: SELECT max(temp_lo) FROM weather; max ----- 46 (1 row) If we wanted to know what city (or cities) that reading occurred in, we might try: SELECT city FROM weather WHERE temp_lo = max(temp_lo); WRONG but this will not work since the aggregate  max  cannot be used in the  WHERE  clause. (This restriction exists because the  WHERE  clause determines which rows will be included in the aggregate calculation; so obviously it has to be evaluated before aggregate functions are computed.) However, as is o...

SQL for Various Data Science Languages : UCDAVIS

  SQL for Various Data Science Languages In this class we’ve gone over relational databases and how SQL is used to retrieve data from them. However, because of the popularity and versatility of SQL, SQL is also used for many big data applications. Below are a few resources for how SQL is used with common big data and data science languages. SQL for R SQLDF Package Documentation Examples SQL for Spark Overview Documentation SQL with Hadoop Hive Overview Documentation SQL for Python Python-SQL Package Documentation

Big Data Adoption and Planning Considerations

Image
 https://www.informit.com/articles/article.aspx?p=2473128&seqNum=11&ranMID=24808 Big Data Analytics Lifecycle Big Data analysis differs from traditional data analysis primarily due to the volume, velocity and variety characteristics of the data being processes. To address the distinct requirements for performing analysis on Big Data, a step-by-step methodology is needed to organize the activities and tasks involved with acquiring, processing, analyzing and repurposing data. The upcoming sections explore a specific data analytics lifecycle that organizes and manages the tasks and activities associated with the analysis of Big Data. From a Big Data adoption and planning perspective, it is important that in addition to the lifecycle, consideration be made for issues of training, education, tooling and staffing of a data analytics team. The Big Data analytics lifecycle can be divided into the following nine stages, as shown in  Figure 3.6 : Business Case Evaluation Data Id...