Udemy - Data Engineering using Kafka and Spark Structured Streaming

File Name

Size

11. Incremental Loads using Spark Structured Streaming/14. Using maxFilerPerTrigger and latestFirst.mp4

98 MB

1. Introduction/1. Introduction.srt

7.5 kB

1. Introduction/2. Important Note for first time Data Engineering Customers.mp4

22 MB

1. Introduction/2. Important Note for first time Data Engineering Customers.srt

5.4 kB

1. Introduction/3. Important Note for Data Engineering Essentials (Python and Spark) Customers.mp4

22 MB

1. Introduction/3. Important Note for Data Engineering Essentials (Python and Spark) Customers.srt

5.1 kB

1. Introduction/4. How to get 30 days complementary lab access.mp4

19 MB

1. Introduction/4. How to get 30 days complementary lab access.srt

4.9 kB

2. Setting up Environment using AWS Cloud9/1. Getting Started with Cloud9.mp4

20 MB

2. Setting up Environment using AWS Cloud9/1. Getting Started with Cloud9.srt

4.9 kB

2. Setting up Environment using AWS Cloud9/2. Creating Cloud9 Environment.mp4

28 MB

2. Setting up Environment using AWS Cloud9/2. Creating Cloud9 Environment.srt

7.7 kB

2. Setting up Environment using AWS Cloud9/3. Warming up with Cloud9 IDE.mp4

17 MB

2. Setting up Environment using AWS Cloud9/3. Warming up with Cloud9 IDE.srt

5.8 kB

2. Setting up Environment using AWS Cloud9/4. Overview of EC2 related to Cloud9.mp4

9.4 MB

2. Setting up Environment using AWS Cloud9/4. Overview of EC2 related to Cloud9.srt

2.7 kB

2. Setting up Environment using AWS Cloud9/5. Opening ports for Cloud9 Instance.mp4

18 MB

2. Setting up Environment using AWS Cloud9/5. Opening ports for Cloud9 Instance.srt

5.3 kB

2. Setting up Environment using AWS Cloud9/6. Associating Elastic IPs to Cloud9 Instance.mp4

29 MB

2. Setting up Environment using AWS Cloud9/6. Associating Elastic IPs to Cloud9 Instance.srt

6.6 kB

2. Setting up Environment using AWS Cloud9/7. Increase EBS Volume Size of Cloud9 Instance.mp4

18 MB

2. Setting up Environment using AWS Cloud9/7. Increase EBS Volume Size of Cloud9 Instance.srt

5.6 kB

2. Setting up Environment using AWS Cloud9/8. Setup Jupyter Lab on Cloud9.mp4

42 MB

2. Setting up Environment using AWS Cloud9/8. Setup Jupyter Lab on Cloud9.srt

12 kB

2. Setting up Environment using AWS Cloud9/9. [Commands] Setup Jupyter Lab on Cloud9.html

615 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/1. Signing up for GCP.mp4

24 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/1. Signing up for GCP.srt

5.5 kB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/1.1 Signing up for GCP.html

146 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/2. Overview of GCP Web Console.mp4

14 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/2. Overview of GCP Web Console.srt

3.5 kB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/2.1 Understanding GCP Web Console.html

158 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/3. Overview of GCP Pricing.mp4

44 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/3. Overview of GCP Pricing.srt

9.1 kB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/3.1 Overview of GCP Pricing.html

151 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/4. Provision Ubuntu VM from GCP.mp4

54 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/4. Provision Ubuntu VM from GCP.srt

13 kB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/4.1 Provision Ubuntu 18.04 Virtual Machine.html

153 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/5. Setup Docker.mp4

75 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/5. Setup Docker.srt

12 kB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/5.1 Setup Docker.html

140 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/6. Validating Python.mp4

34 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/6. Validating Python.srt

6.0 kB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/6.1 Validating Python.html

145 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/7. Setup Jupyter Lab.mp4

79 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/7. Setup Jupyter Lab.srt

16 kB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/7.1 Setup Jupyter Lab.html

145 B

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/8. Setup Jupyter Lab locally on Mac.mp4

39 MB

3. Setting up Environment - Overview of GCP and Provision Ubuntu VM/8. Setup Jupyter Lab locally on Mac.srt

9.8 kB

4. Setup Single Node Hadoop Cluster/1. Introduction to Single Node Hadoop Cluster.mp4

16 MB

4. Setup Single Node Hadoop Cluster/1. Introduction to Single Node Hadoop Cluster.srt

5.0 kB

4. Setup Single Node Hadoop Cluster/2. Setup Prerequisites.mp4

32 MB

4. Setup Single Node Hadoop Cluster/2. Setup Prerequisites.srt

8.4 kB

4. Setup Single Node Hadoop Cluster/3. Setup Password less login.mp4

17 MB

4. Setup Single Node Hadoop Cluster/3. Setup Password less login.srt

6.4 kB

4. Setup Single Node Hadoop Cluster/4. Download and Install Hadoop.mp4

31 MB

4. Setup Single Node Hadoop Cluster/4. Download and Install Hadoop.srt

7.5 kB

4. Setup Single Node Hadoop Cluster/5. Configure Hadoop HDFS.mp4

47 MB

4. Setup Single Node Hadoop Cluster/5. Configure Hadoop HDFS.srt

12 kB

4. Setup Single Node Hadoop Cluster/6. Start and Validate HDFS.mp4

46 MB

4. Setup Single Node Hadoop Cluster/6. Start and Validate HDFS.srt

10 kB

4. Setup Single Node Hadoop Cluster/7. Configure Hadoop YARN.mp4

7.7 MB

4. Setup Single Node Hadoop Cluster/7. Configure Hadoop YARN.srt

2.0 kB

4. Setup Single Node Hadoop Cluster/8. Start and Validate YARN.mp4

14 MB

4. Setup Single Node Hadoop Cluster/8. Start and Validate YARN.srt

3.3 kB

4. Setup Single Node Hadoop Cluster/9. Managing Single Node Hadoop.mp4

20 MB

4. Setup Single Node Hadoop Cluster/9. Managing Single Node Hadoop.srt

7.0 kB

5. Setup Hive and Spark/1. Setup Data Sets for Practice.mp4

39 MB

5. Setup Hive and Spark/1. Setup Data Sets for Practice.srt

7.4 kB

5. Setup Hive and Spark/2. Download and Install Hive.mp4

13 MB

5. Setup Hive and Spark/2. Download and Install Hive.srt

3.8 kB

5. Setup Hive and Spark/3. Setup Database for Hive Metastore.mp4

53 MB

5. Setup Hive and Spark/3. Setup Database for Hive Metastore.srt

12 kB

5. Setup Hive and Spark/4. Configure and Setup Hive Metastore.mp4

54 MB

5. Setup Hive and Spark/4. Configure and Setup Hive Metastore.srt

12 kB

5. Setup Hive and Spark/5. Launch and Validate Hive.mp4

36 MB

5. Setup Hive and Spark/5. Launch and Validate Hive.srt

8.0 kB

5. Setup Hive and Spark/6. Scripts to Manage Single Node Cluster.mp4

22 MB

5. Setup Hive and Spark/6. Scripts to Manage Single Node Cluster.srt

7.7 kB

5. Setup Hive and Spark/7. Download and Install Spark 2.mp4

25 MB

5. Setup Hive and Spark/7. Download and Install Spark 2.srt

5.4 kB

5. Setup Hive and Spark/8. Configure Spark 2.mp4

67 MB

5. Setup Hive and Spark/8. Configure Spark 2.srt

14 kB

5. Setup Hive and Spark/9. Validate Spark 2 using CLIs.mp4

66 MB

5. Setup Hive and Spark/9. Validate Spark 2 using CLIs.srt

13 kB

5. Setup Hive and Spark/10. Validate Jupyter Lab Setup.mp4

68 MB

5. Setup Hive and Spark/10. Validate Jupyter Lab Setup.srt

16 kB

5. Setup Hive and Spark/11. Integrate Spark 2 with Jupyter Lab.mp4

39 MB

5. Setup Hive and Spark/11. Integrate Spark 2 with Jupyter Lab.srt

9.6 kB

5. Setup Hive and Spark/12. Download and Install Spark 3.mp4

15 MB

5. Setup Hive and Spark/12. Download and Install Spark 3.srt

3.1 kB

5. Setup Hive and Spark/13. Configure Spark 3.mp4

59 MB

5. Setup Hive and Spark/13. Configure Spark 3.srt

10 kB

5. Setup Hive and Spark/14. Validate Spark 3 using CLIs.mp4

62 MB

5. Setup Hive and Spark/14. Validate Spark 3 using CLIs.srt

13 kB

5. Setup Hive and Spark/15. Integrate Spark 3 with Jupyter Lab.mp4

25 MB

5. Setup Hive and Spark/15. Integrate Spark 3 with Jupyter Lab.srt

7.0 kB

6. Setup Single Node Kafka Cluster/1. Download and Install Kafka.mp4

36 MB

6. Setup Single Node Kafka Cluster/1. Download and Install Kafka.srt

8.3 kB

6. Setup Single Node Kafka Cluster/2. Configure and Start Zookeeper.mp4

29 MB

6. Setup Single Node Kafka Cluster/2. Configure and Start Zookeeper.srt

5.6 kB

6. Setup Single Node Kafka Cluster/3. Configure and Start Kafka Broker.mp4

42 MB

6. Setup Single Node Kafka Cluster/3. Configure and Start Kafka Broker.srt

8.6 kB

6. Setup Single Node Kafka Cluster/4. Scripts to manage single node cluster.mp4

26 MB

6. Setup Single Node Kafka Cluster/4. Scripts to manage single node cluster.srt

7.6 kB

6. Setup Single Node Kafka Cluster/5. Overview of Kafka CLI.mp4

63 MB

6. Setup Single Node Kafka Cluster/5. Overview of Kafka CLI.srt

14 kB

6. Setup Single Node Kafka Cluster/6. Setup Retail log Generator.mp4

32 MB

6. Setup Single Node Kafka Cluster/6. Setup Retail log Generator.srt

6.8 kB

6. Setup Single Node Kafka Cluster/7. Redirecting logs to Kafka.mp4

48 MB

6. Setup Single Node Kafka Cluster/7. Redirecting logs to Kafka.srt

9.1 kB

7. Getting Started with Kafka/1. Overview of Kafka.mp4

41 MB

7. Getting Started with Kafka/1. Overview of Kafka.srt

9.3 kB

7. Getting Started with Kafka/2. Managing Topics using Kafka CLI.mp4

48 MB

7. Getting Started with Kafka/2. Managing Topics using Kafka CLI.srt

12 kB

7. Getting Started with Kafka/3. Produce and Consume Messages using CLI.mp4

27 MB

7. Getting Started with Kafka/3. Produce and Consume Messages using CLI.srt

6.7 kB

7. Getting Started with Kafka/4. Validate Generation of Web Server Logs.mp4

19 MB

7. Getting Started with Kafka/4. Validate Generation of Web Server Logs.srt

3.9 kB

7. Getting Started with Kafka/5. Create Web Server using nc.mp4

16 MB

7. Getting Started with Kafka/5. Create Web Server using nc.srt

5.1 kB

7. Getting Started with Kafka/6. Produce retail logs to Kafka Topic.mp4

15 MB

7. Getting Started with Kafka/6. Produce retail logs to Kafka Topic.srt

3.7 kB

7. Getting Started with Kafka/7. Consume retail logs from Kafka Topic.mp4

14 MB

7. Getting Started with Kafka/7. Consume retail logs from Kafka Topic.srt

2.7 kB

7. Getting Started with Kafka/8. Clean up Kafka CLI Sessions to produce and consume messages.mp4

10 MB

7. Getting Started with Kafka/8. Clean up Kafka CLI Sessions to produce and consume messages.srt

2.3 kB

7. Getting Started with Kafka/9. Define Kafka Connect to produce.mp4

60 MB

7. Getting Started with Kafka/9. Define Kafka Connect to produce.srt

13 kB

7. Getting Started with Kafka/10. Validate Kafka Connect to produce.mp4

60 MB

7. Getting Started with Kafka/10. Validate Kafka Connect to produce.srt

12 kB

8. Data Ingestion using Kafka Connect/1. Overview of Kafka Connect.mp4

9.8 MB

8. Data Ingestion using Kafka Connect/1. Overview of Kafka Connect.srt

2.7 kB

8. Data Ingestion using Kafka Connect/2. Define Kafka Connect to Produce Messages.mp4

69 MB

8. Data Ingestion using Kafka Connect/2. Define Kafka Connect to Produce Messages.srt

15 kB

8. Data Ingestion using Kafka Connect/3. Validate Kafka Connect to produce messages.mp4

64 MB

8. Data Ingestion using Kafka Connect/3. Validate Kafka Connect to produce messages.srt

12 kB

8. Data Ingestion using Kafka Connect/4. Cleanup Kafka Connect to produce messages.mp4

20 MB

8. Data Ingestion using Kafka Connect/4. Cleanup Kafka Connect to produce messages.srt

4.3 kB

8. Data Ingestion using Kafka Connect/5. Write Data to HDFS using Kafka Connect.mp4

8.1 MB

8. Data Ingestion using Kafka Connect/5. Write Data to HDFS using Kafka Connect.srt

2.4 kB

8. Data Ingestion using Kafka Connect/6. Setup HDFS 3 Sink Connector Plugin.mp4

64 MB

8. Data Ingestion using Kafka Connect/6. Setup HDFS 3 Sink Connector Plugin.srt

11 kB

8. Data Ingestion using Kafka Connect/7. Overview of Kafka Consumer Groups.mp4

59 MB

8. Data Ingestion using Kafka Connect/7. Overview of Kafka Consumer Groups.srt

13 kB

8. Data Ingestion using Kafka Connect/8. Configure HDFS 3 Sink Properties.mp4

53 MB

8. Data Ingestion using Kafka Connect/8. Configure HDFS 3 Sink Properties.srt

11 kB

8. Data Ingestion using Kafka Connect/9. Run and Validate HDFS 3 Sink.mp4

56 MB

8. Data Ingestion using Kafka Connect/9. Run and Validate HDFS 3 Sink.srt

12 kB

8. Data Ingestion using Kafka Connect/10. Cleanup Kafka Connect to consume messages.mp4

25 MB

8. Data Ingestion using Kafka Connect/10. Cleanup Kafka Connect to consume messages.srt

6.2 kB

9. Overview of Spark Structured Streaming/1. Understanding Streaming Context.mp4

39 MB

9. Overview of Spark Structured Streaming/1. Understanding Streaming Context.srt

9.0 kB

9. Overview of Spark Structured Streaming/2. Validate Log Data for Streaming.mp4

16 MB

9. Overview of Spark Structured Streaming/2. Validate Log Data for Streaming.srt

3.4 kB

9. Overview of Spark Structured Streaming/3. Push log messages to Netcat Webserver.mp4

28 MB

9. Overview of Spark Structured Streaming/3. Push log messages to Netcat Webserver.srt

7.5 kB

9. Overview of Spark Structured Streaming/4. Overview of built-in Input Sources.mp4

18 MB

9. Overview of Spark Structured Streaming/4. Overview of built-in Input Sources.srt

3.7 kB

9. Overview of Spark Structured Streaming/5. Reading Web Server logs using Spark Structured Streaming.mp4

75 MB

9. Overview of Spark Structured Streaming/5. Reading Web Server logs using Spark Structured Streaming.srt

19 kB

9. Overview of Spark Structured Streaming/6. Overview of Output Modes.mp4

8.7 MB

9. Overview of Spark Structured Streaming/6. Overview of Output Modes.srt

1.9 kB

9. Overview of Spark Structured Streaming/7. Using append as Output Mode.mp4

65 MB

9. Overview of Spark Structured Streaming/7. Using append as Output Mode.srt

12 kB

9. Overview of Spark Structured Streaming/8. Using complete as Output Mode.mp4

53 MB

9. Overview of Spark Structured Streaming/8. Using complete as Output Mode.srt

11 kB

9. Overview of Spark Structured Streaming/9. Using update as Output Mode.mp4

24 MB

9. Overview of Spark Structured Streaming/9. Using update as Output Mode.srt

6.9 kB

9. Overview of Spark Structured Streaming/10. Overview of Triggers in Spark Structured Streaming.mp4

39 MB

9. Overview of Spark Structured Streaming/10. Overview of Triggers in Spark Structured Streaming.srt

9.3 kB

9. Overview of Spark Structured Streaming/11. Overview of built-in Output Sinks.mp4

19 MB

9. Overview of Spark Structured Streaming/11. Overview of built-in Output Sinks.srt

4.3 kB

9. Overview of Spark Structured Streaming/12. Previewing the Streaming Data.mp4

63 MB

9. Overview of Spark Structured Streaming/12. Previewing the Streaming Data.srt

12 kB

10. Kafka and Spark Structured Streaming Integration/1. Create Kafka Topic.mp4

18 MB

10. Kafka and Spark Structured Streaming Integration/1. Create Kafka Topic.srt

4.9 kB

10. Kafka and Spark Structured Streaming Integration/2. Read Data from Kafka Topic.mp4

42 MB

10. Kafka and Spark Structured Streaming Integration/2. Read Data from Kafka Topic.srt

9.3 kB

10. Kafka and Spark Structured Streaming Integration/3. Preview data using console.mp4

37 MB

10. Kafka and Spark Structured Streaming Integration/3. Preview data using console.srt

8.3 kB

10. Kafka and Spark Structured Streaming Integration/4. Preview data using memory.mp4

28 MB

10. Kafka and Spark Structured Streaming Integration/4. Preview data using memory.srt

5.4 kB

10. Kafka and Spark Structured Streaming Integration/5. Transform Data using Spark APIs.mp4

82 MB

10. Kafka and Spark Structured Streaming Integration/5. Transform Data using Spark APIs.srt

15 kB

10. Kafka and Spark Structured Streaming Integration/6. Write Data to HDFS using Spark.mp4

77 MB

10. Kafka and Spark Structured Streaming Integration/6. Write Data to HDFS using Spark.srt

12 kB

10. Kafka and Spark Structured Streaming Integration/7. Validate Data in HDFS using Spark.mp4

67 MB

10. Kafka and Spark Structured Streaming Integration/7. Validate Data in HDFS using Spark.srt

11 kB

10. Kafka and Spark Structured Streaming Integration/8. Write Data to HDFS using Spark using Header.mp4

65 MB

10. Kafka and Spark Structured Streaming Integration/8. Write Data to HDFS using Spark using Header.srt

11 kB

10. Kafka and Spark Structured Streaming Integration/9. Cleanup Kafka Connect and Files in HDFS.mp4

32 MB

10. Kafka and Spark Structured Streaming Integration/9. Cleanup Kafka Connect and Files in HDFS.srt

4.7 kB

11. Incremental Loads using Spark Structured Streaming/1. Overview of Spark Structured Streaming Triggers.mp4

9.4 MB

11. Incremental Loads using Spark Structured Streaming/1. Overview of Spark Structured Streaming Triggers.srt

2.6 kB

11. Incremental Loads using Spark Structured Streaming/2. Steps for Incremental Data Processing.mp4

23 MB

11. Incremental Loads using Spark Structured Streaming/2. Steps for Incremental Data Processing.srt

5.9 kB

11. Incremental Loads using Spark Structured Streaming/3. Create Working Directory in HDFS.mp4

16 MB

11. Incremental Loads using Spark Structured Streaming/3. Create Working Directory in HDFS.srt

5.1 kB

11. Incremental Loads using Spark Structured Streaming/4. Logic to Upload GHArchive Files.mp4

82 MB

11. Incremental Loads using Spark Structured Streaming/4. Logic to Upload GHArchive Files.srt

20 kB

11. Incremental Loads using Spark Structured Streaming/5. Upload GHArchive Files to HDFS.mp4

93 MB

11. Incremental Loads using Spark Structured Streaming/5. Upload GHArchive Files to HDFS.srt

21 kB

11. Incremental Loads using Spark Structured Streaming/6. Add new GHActivity JSON Files.mp4

33 MB

11. Incremental Loads using Spark Structured Streaming/6. Add new GHActivity JSON Files.srt

6.3 kB

11. Incremental Loads using Spark Structured Streaming/7. Read JSON Data using Spark Structured streaming.mp4

46 MB

11. Incremental Loads using Spark Structured Streaming/7. Read JSON Data using Spark Structured streaming.srt

11 kB

11. Incremental Loads using Spark Structured Streaming/8. Write in Parquet File Format.mp4

12 MB

11. Incremental Loads using Spark Structured Streaming/8. Write in Parquet File Format.srt

3.2 kB

11. Incremental Loads using Spark Structured Streaming/9. Analyze GHArchive Data in Parquet files using Spark.mp4

50 MB

11. Incremental Loads using Spark Structured Streaming/9. Analyze GHArchive Data in Parquet files using Spark.srt

12 kB

11. Incremental Loads using Spark Structured Streaming/10. Add New GHActivity JSON files.mp4

13 MB

11. Incremental Loads using Spark Structured Streaming/10. Add New GHActivity JSON files.srt

2.4 kB

11. Incremental Loads using Spark Structured Streaming/11. Load Data Incrementally to Target Table.mp4

69 MB

11. Incremental Loads using Spark Structured Streaming/11. Load Data Incrementally to Target Table.srt

12 kB

11. Incremental Loads using Spark Structured Streaming/12. Validate Incremental Load.mp4

20 MB

11. Incremental Loads using Spark Structured Streaming/12. Validate Incremental Load.srt

4.6 kB

11. Incremental Loads using Spark Structured Streaming/13. Add New GHActivity JSON files.mp4

8.0 MB

11. Incremental Loads using Spark Structured Streaming/13. Add New GHActivity JSON files.srt

1.9 kB

1. Introduction/1. Introduction.mp4

31 MB

11. Incremental Loads using Spark Structured Streaming/14. Using maxFilerPerTrigger and latestFirst.srt

17 kB

11. Incremental Loads using Spark Structured Streaming/15. Validate Incremental Load.mp4

22 MB

11. Incremental Loads using Spark Structured Streaming/15. Validate Incremental Load.srt

5.4 kB

11. Incremental Loads using Spark Structured Streaming/16. Add New GHActivity JSON files.mp4

10 MB

11. Incremental Loads using Spark Structured Streaming/16. Add New GHActivity JSON files.srt

2.4 kB

11. Incremental Loads using Spark Structured Streaming/17. Incremental Load using Archival Process.mp4

92 MB

11. Incremental Loads using Spark Structured Streaming/17. Incremental Load using Archival Process.srt

17 kB

11. Incremental Loads using Spark Structured Streaming/18. Validate Incremental Load.mp4

23 MB

11. Incremental Loads using Spark Structured Streaming/18. Validate Incremental Load.srt

5.0 kB