1. ã¯ããã«
è¿å¹ŽãããŒã¿ã®åãæ±ããå¢ããäžã§ããã®ããŒã¿ãå¹æçã«åŠçã»åæããæè¡ãæ±ããããŠããŸãããã®äžã§ãClouderaèªå®ãApache Hadoopãšããèšèãè³ã«ããããšãå¢ããŠããã®ã§ã¯ãªãã§ããããããã®ã»ã¯ã·ã§ã³ã§ã¯ããããã®ããŒã¯ãŒãã®åºæ¬çãªæå³ãèæ¯ã解説ããŸãã
1.1 Clouderaèªå®ãšã¯
Clouderaèªå®ã¯ãããŒã¿ãšã³ãžãã¢ãªã³ã°ãããŒã¿åæã®åéã§ã®å°éç¥èãæè¡ãæã€ããšã蚌æããè³æ Œã®äžã€ã§ãããã®èªå®ãååŸããããšã§ãããŒã¿ã®åãæ±ãã«é¢ããé«åºŠãªã¹ãã«ãç¥èãæã£ãŠããããšã第äžè ã«èªããããã®ã§ããç¹ã«ã倧æäŒæ¥ãå°éçãªãããžã§ã¯ãã§ã®æ¡çšã®éã«ããã®è³æ Œã¯å€§ããªã¢ããã³ããŒãžãšãªããŸãã
1.2 Apache Hadoopã®éèŠæ§
Apache Hadoopã¯ã倧éã®ããŒã¿ãå¹æçã«åŠçããããã®ãªãŒãã³ãœãŒã¹ã®ãã¬ãŒã ã¯ãŒã¯ã§ãããã®æè¡ã¯ãããã°ããŒã¿ã®æ代ã«ãããŠãããŒã¿ã®ä¿åãåæãé«éã«è¡ãããã®åºç€ãšããŠåºãå©çšãããŠããŸããHadoopã®ç¹åŸŽãšããŠãè€æ°ã®ãã·ã³ã«ããŒã¿ãåæ£ãããŠåŠçããããšãã§ããç¹ãæããããŸããããã«ãããåŸæ¥ã®æ¹æ³ã§ã¯é£ããã£ã倧èŠæš¡ãªããŒã¿ã®åŠçãå¯èœãšãªããŸããã
ãã®ããã«ãClouderaèªå®ãšApache Hadoopã¯ãçŸä»£ã®ããŒã¿ããªãã³ãªç€ŸäŒã«ãããŠãéåžžã«éèŠãªäœçœ®ãå ããŠããŸãã次ã®ã»ã¯ã·ã§ã³ã§ã¯ãClouderaèªå®ã®è©³çŽ°ã«ã€ããŠè©³ãã解説ããŠãããŸãã
2. Clouderaèªå®ã®è©³çŽ°
è¿å¹ŽãããŒã¿ã®åãæ±ããå¢ããäžã§ããã®ããŒã¿ãå¹æçã«åŠçã»åæããæè¡ãæ±ããããŠããŸãããã®äžã§ãClouderaèªå®ã¯ããŒã¿ãšã³ãžãã¢ãªã³ã°ãããŒã¿åæã®åéã§ã®å°éç¥èã蚌æããããã®éèŠãªè³æ Œãšãªã£ãŠããŸãããã®ã»ã¯ã·ã§ã³ã§ã¯ãClouderaèªå®ã®è©³çŽ°ã«ã€ããŠãããæ·±ãæãäžããŠè§£èª¬ããŸãã
2.1 èªå®ã®çš®é¡ãšãã®ç¹åŸŽ
Clouderaã¯ãApache Hadoopã®å®çšçãªç¥èã蚌æããããã®Cloudera Certified Developer for Apache Hadoop (CCDH)ãšããèªå®ãæäŸããŠããŸãããã®èªå®ã¯ãHadoopã®åºæ¬çãªã³ã³ã»ãããHadoop Distributed File System (HDFS)ãMapReduceã®å éšæ§é ã«é¢ããç¥èãè©äŸ¡ããŸãããŸããClouderaã¯ä»ã«ãããŒã¿ãµã€ãšã³ãã£ã¹ãã管çè åãã®èªå®ãæäŸããŠãããããããã®åœ¹å²ã«å¿ããå°éç¥èã蚌æããããšãã§ããŸãããããã®èªå®ã¯ãããŒã¿é¢é£ã®è·çš®ã«ãããŠãå°é家ãšããŠã®ã¹ãã«ãç¥èã第äžè ã«èªããããããã®ãã®ãšãªã£ãŠããŸãã
2.2 éçºè åãèªå®ã®ç®ç
ãã®èªå®ã®äž»ãªç®çã¯ãApache Hadoopã䜿çšããŠå ç¢ãªããŒã¿åŠçã¢ããªã±ãŒã·ã§ã³ãäœæããèœåãæã€ããšã蚌æããããšã§ããHadoopã¯ãããã°ããŒã¿ã®æ代ã«ãããŠãããŒã¿ã®ä¿åãåæãé«éã«è¡ãããã®åºç€ãšããŠåºãå©çšãããŠããŸãããã®èªå®ãååŸããããšã§ãHadoopã®ãšã³ã·ã¹ãã ãããžã§ã¯ãã掻çšããæ¹æ³ããMapReduceã§ã®ããŒã¿ã»ããã®ãªã³ã¯æ¹æ³ãªã©ãå®éã®ããŒã¿åæã«å¿ èŠãªé«åºŠãªHadoop APIãããã¯ãç¿åŸããŠããããšã蚌æãããŸãããŸãããã®èªå®ã¯ãããŒã¿ãšã³ãžãã¢ãªã³ã°ã®åéã§ã®ãã£ãªã¢ã¢ãããç®æã人ã ã«ãšã£ãŠã倧ããªã¢ããã³ããŒãžãšãªãã§ãããã
2.3 è©Šéšã®åœ¢åŒãšå 容
CCDHã®è©Šéšã¯ã55ã®è³ªåãããªãã90å以å ã«å®äºããå¿ èŠããããŸããåæ Œåºæºã¯70%以äžã§ããè©Šéšã®å 容ã¯ãMapReduceã®å éšãHDFSãMapReduceã³ãŒãã®èšè¿°æ¹æ³ãªã©ãHadoopã®åºæ¬çãªã³ã³ã»ããã«é¢ãããã®ãäžå¿ãšãªããŸãããŸããHiveãPigãSqoopãFlumeãOozieãªã©ã®Hadoopãšã³ã·ã¹ãã ãããžã§ã¯ãã®æŽ»çšæ¹æ³ãè©Šéšã®ç¯å²ã«å«ãŸããŸãããã®è©ŠéšãéããŠãåéšè ã®Hadoopã«é¢ããå®è·µçãªç¥èãæè¡ãè©äŸ¡ãããŸããè©Šéšã®æºåãšããŠã¯ãå®éã®æ¥åçµéšãå°éçãªãã¬ãŒãã³ã°ãæšå¥šãããŠããŸãã
ãã®ããã«ãClouderaèªå®ã¯ãçŸä»£ã®ããŒã¿ããªãã³ãªç€ŸäŒã«ãããŠãããŒã¿ãšã³ãžãã¢ãªã³ã°ãããŒã¿åæã®åéã§ã®å°éç¥èã蚌æããããã®éèŠãªè³æ Œãšãªã£ãŠããŸãã次ã®ã»ã¯ã·ã§ã³ã§ã¯ãClouderaèªå®ã®é£æ床ãè©Šéšã®ãã€ã³ãã«ã€ããŠè©³ãã解説ããŠãããŸãã
3. é£æ床ã«ã€ããŠ
3.1 è©Šéšã®é£æ床ã®è©äŸ¡
Cloudera Certified Developer for Apache Hadoop (CCDH)ã®è©Šéšã¯ãApache Hadoopã䜿çšããŠå ç¢ãªããŒã¿åŠçã¢ããªã±ãŒã·ã§ã³ãäœæããã¹ãã«ãè©äŸ¡ãããã®ã§ãããã®è©Šéšã¯ãMapReduceãHadoop Distributed File System (HDFS)ã®å éšæ§é ãããã³MapReduceã³ãŒãã®èšè¿°æ¹æ³ã«é¢ããç¥èãäžå¿ã«æ§ç¯ãããŠããŸããããã«ãHadoopã®éçºããããã°ãã¯ãŒã¯ãããŒã®å®è£ ãããã³äžè¬çãªã¢ã«ãŽãªãºã ã«é¢ãããã¹ããã©ã¯ãã£ã¹ãã«ããŒãããŠããŸãããã®ãããªå 容ãè©Šéšã«åãå ¥ããããŠãããããåéšè ã¯Hadoopã®åºæ¬ããå¿çšãŸã§ã®å¹ åºãç¥èãæ±ããããŸããHadoopã¯ãããã°ããŒã¿ã®åŠçã«é¢ããæè¡ãšããŠãè¿å¹Žéåžžã«æ³šç®ãããŠãããå€ãã®äŒæ¥ãå°å ¥ãé²ããŠããŸãããã®ãããHadoopã«é¢ããç¥èãã¹ãã«ã¯ãçŸä»£ã®ITæ¥çã§éåžžã«äŸ¡å€ããããšèšããŸãã
3.2 åæ Œçãšãã®èæ¯
CCDHã®è©Šéšã¯ã55ã®è³ªåããæãç«ã£ãŠããã90å以å ã«å®äºããå¿ èŠããããŸããåæ Œåºæºã¯70%以äžã§ãããã®è©Šéšã¯å®éã®Hadoopéçºè ãçŽé¢ããçŸå®ã®èª²é¡ã«åããããã®ãã®ã§ãããFirebrand Trainingã®ãããªãã¬ãŒãã³ã°ãããã€ããŒã¯ãåéšè ããã®è©Šéšã«åæ Œããããã®ååãªæºåãæäŸããŸãããããããã®è©Šéšã®åæ Œçã¯ãä»ã®äžè¬çãªITèªå®è©Šéšãšæ¯èŒããŠãé«ããšã¯èšããŸããããã®èæ¯ã«ã¯ãHadoopã®ãšã³ã·ã¹ãã ãéåžžã«åºããå€å²ã«ããããããåéšè ãå šãŠã®ãããã¯ã«ç²ŸéããŠããããšãé£ãããšããäºæ ããããŸãããŸããHadoopã¯ãªãŒãã³ãœãŒã¹ã®ãããžã§ã¯ãã§ããããã®æè¡ãããŒã«ã¯æ¥ã é²åããŠããŸãããã®ãããåéšè ã¯åžžã«ææ°ã®æ å ±ããã£ããã¢ããããå¿ èŠããããŸãã
3.3 ä»ã®ITèªå®è©Šéšãšã®æ¯èŒ
CCDHã¯ãApache Hadoopã®éçºã«ç¹åããèªå®è©Šéšã§ããããã®å 容ãšé£æ床ã¯ãä»ã®äžè¬çãªITèªå®è©Šéšãšã¯ç°ãªããŸããäŸãã°ãJavaãä»ã®ããã°ã©ãã³ã°èšèªã®èªå®è©Šéšã¯ãç¹å®ã®èšèªã®ç¥èãã¹ãã«ãè©äŸ¡ããã®ã«å¯ŸããCCDHã¯ã倧èŠæš¡ãªããŒã¿ã»ãããå¹æçã«åŠçããããã®ç¹å®ã®æè¡ãããŒã«ã«çŠç¹ãåœãŠãŠããŸãããã®ãããCCDHã®è©Šéšå 容ã¯ãå®éã®æ¥çã®ããŒãºããã¬ã³ãã«å¯æ¥ã«é¢é£ããŠãããä»ã®è©Šéšãããå®è·µçãªã¹ãã«ã匷調ããŠããŸããããã«ãHadoopã®ãšã³ã·ã¹ãã ã®è€éããããã®æè¡çãªæ·±ããèæ ®ãããšãä»ã®ITèªå®è©Šéšãšæ¯èŒããŠãCCDHã®é£æ床ã¯é«ããšèšããã§ãããããããããã®è©ŠéšãéããŠãåéšè ã¯Hadoopã®æ·±ãç¥èãã¹ãã«ã身ã«ã€ããããšãã§ããããã°ããŒã¿ã®åŠçã«é¢ããå°é家ãšããŠã®å°äœã確ç«ããããšãã§ããŸãã
4. è©Šéšã®ãã€ã³ã
ITèªå®è©Šéšã¯ãæè¡çãªã¹ãã«ãšç¥èãè©äŸ¡ããããã®ãã®ã§ããããããè©Šéšã®æåã¯ãåã«ç¥èãæã£ãŠããã ãã§ã¯äžååã§ããå¹æçãªå匷æ³ãè©Šéšã®æ§é ãžã®ç解ããããŠè©Šéšåœæ¥ã®é©åãªå¯Ÿçãå¿ èŠã§ãã
4.1 éèŠãªãããã¯ãšãã®ç解
Cloudera Certified Developer for Apache Hadoop (CCDH)ã®è©Šéšã¯ãApache Hadoopã®éçºã«é¢ããå¹ åºããããã¯ãã«ããŒããŠããŸããMapReduceã®å éšæ§é ãHadoop Distributed File System (HDFS)ã®ä»çµã¿ãããã«ã¯HiveãPigãªã©ã®Hadoopãšã³ã·ã¹ãã ã®ãããžã§ã¯ãã®å©çšæ¹æ³ãªã©ãå€å²ã«ãããå 容ãå«ãŸããŠããŸãããããã®ãããã¯ãæ·±ãç解ããããšã§ãè©Šéšã®åé¡ã«å¹æçã«å¯Ÿå¿ããããšãã§ããŸãã
ç¹ã«ãMapReduceã®ã¢ã«ãŽãªãºã ãHDFSã®ããŒã¿é 眮ã®ä»çµã¿ã¯ãè©Šéšã®äžå¿çãªãããã¯ãšãªã£ãŠããŸãããããã®åºæ¬çãªæŠå¿µããã£ãããšç解ããå®éã®éçºç°å¢ã§ã®å¿çšæ¹æ³ãåŠã¶ããšããè©Šéšã®æåã®éµãšãªããŸãã
4.2 å¹æçãªå匷æ³
å¹æçãªå匷æ³ã¯ãå人ã®åŠç¿ã¹ã¿ã€ã«ãçµéšã«å¿ããŠç°ãªããŸããããããå®è·µçãªææ³ãåãå ¥ããããšã¯éåžžã«æå¹ã§ããå ·äœçã«ã¯ãå®éã®ããŒã¿ã»ããã䜿çšããŠHadoopã¢ããªã±ãŒã·ã§ã³ãéçºããç·Žç¿ãè¡ã£ãããéå»ã®è©Šéšåé¡ã解ãããšã§ãè©Šéšã®é°å²æ°ãåé¡ã®åŸåãæŽãããšãã§ããŸãã
ãŸãããªã³ã©ã€ã³ã®åŠç¿ãªãœãŒã¹ãæžç±ã掻çšããŠãHadoopã®åºæ¬ããå¿çšãŸã§ã®ç¥èãåºããããšãããããã§ããç¹ã«ãå®éã®éçºçµéšãå°ãªãæ¹ã¯ããã³ãºãªã³ã®ç·Žç¿ãäžå¿ã«å匷ãé²ãããšè¯ãã§ãããã
4.3 è©Šéšåœæ¥ã®å¯Ÿç
è©Šéšåœæ¥ã¯ãååãªäŒæ¯ããšãããšãéèŠã§ãããŸããè©ŠéšäŒå Žã®ã«ãŒã«ãæã¡èŸŒã¿å¯èœãªã¢ã€ãã ã«ã€ããŠäºåã«ç¢ºèªããŠãããšè¯ãã§ããããè©Šéšäžã¯ãæéãé©åã«ç®¡çããªãããçŠãããå·éã«åé¡ã解ãããšãæ±ããããŸãã
æåŸã«ãCloudera Certified Developer for Apache Hadoop (CCDH)ã®è©Šéšã¯ãHadoopã®éçºã«é¢ããæ·±ãç¥èãšã¹ãã«ãè©äŸ¡ãããã®ã§ãããã®è©ŠéšãéããŠãèªèº«ã®ã¹ãã«ã蚌æãããã£ãªã¢ã®ãããªãåäžãç®æããŸãããã
5. å®éã®è©Šéšäœéš
Cloudera Certified Developer for Apache Hadoop (CCDH)ã®è©Šéšã¯ãå€ãã®ITãããã§ãã·ã§ãã«ã«ãšã£ãŠéèŠãªã¹ããããšãªããã®ã§ãããã®ã»ã¯ã·ã§ã³ã§ã¯ãå®éã«è©Šéšãåéšããæ¹ã ã®äœéšè«ãã¢ããã€ã¹ãå ±æããŸãã
5.1 åéšè ã®å£°
å€ãã®åéšè ã¯ãè©Šéšã®é£æ床ãåé¡ã®å 容ã«ã€ããŠã®ææ³ãæã£ãŠããŸããäžéšã®åéšè ã¯ãè©Šéšã®å 容ãå®éã®éçºç°å¢ã§ã®çµéšãšå¯æ¥ã«é¢é£ããŠãããšæããŸãããäžæ¹ãåããŠHadoopã«è§Šããæ¹ã ã¯ãåºæ¬çãªæŠå¿µãçšèªã«ã€ããŠã®åé¡ãææŠçã§ãããšè¿°ã¹ãŠããŸãã
ãŸããè©Šéšã®æé管çã«ã€ããŠããå€ãã®æèŠãå¯ããããŠããŸããç¹ã«ãè€æ°ã®åé¡ã解ãéã®æéé åããé£ããåé¡ãã¹ãããããŠåŸã§æ»ãæŠç¥ãªã©ãæ§ã ãªã¢ãããŒããè©ŠãããŠããŸãã
5.2 åæ Œè ã®ã¢ããã€ã¹
åæ Œããåéšè ããã®ã¢ããã€ã¹ã¯ãéåžžã«äŸ¡å€ã®ãããã®ã§ããå€ãã®åæ Œè ã¯ãå®è·µçãªçµéšãè©Šéšã®æåã«å€§ããå¯äžãããšåŒ·èª¿ããŠããŸããå ·äœçã«ã¯ãå®éã®ããŒã¿ã»ããã䜿çšããŠã®ãããžã§ã¯ãçµéšããHadoopãšã³ã·ã¹ãã ã®åããŒã«ã®äœ¿çšçµéšããè©Šéšã®åé¡ã«å¯Ÿããç解ãæ·±ããã®ã«åœ¹ç«ã€ãšè¿°ã¹ãŠããŸãã
ããã«ãè©Šéšã®åã«ååãªåŸ©ç¿ãšæš¡æ¬è©Šéšãè¡ãããšããããŠè©Šéšåœæ¥ã¯å·éãªå¿æã¡ã§èšãããšããæåã®éµã§ãããšã®æèŠãå€ãå¯ããããŠããŸãã
5.3 äžåæ Œè ã®åçç¹
æ®å¿µãªããè©Šéšã«åæ Œããªãã£ãåéšè ãããŸããããã®çµéšã¯æ¬¡åã®ææŠã«çããããšãã§ããŸããäžåæ Œè ã®äžã«ã¯ãåºæ¬çãªæŠå¿µã®ç解ãäžååã§ãã£ãããç¹å®ã®ãããã¯ã«ã€ããŠã®ç¥èãæµ ãã£ããããããšãåçç¹ãšããŠæããæ¹ã ãããŸãã
ãŸããè©Šéšã®æéé åãåé¡ã®èªã¿åãã«é¢ãã課é¡ããå€ãã®åçç¹ãšããŠææãããŠããŸãããããã®åçãèžãŸãã次åã®è©Šéšã«åããŠã®æºåãé²ããããšããæåãžã®éãšãªãã§ãããã
æåŸã«ãCloudera Certified Developer for Apache Hadoop (CCDH)ã®è©Šéšã¯ãHadoopéçºè ãšããŠã®ã¹ãã«ãšç¥èã蚌æãããã®ã§ãããã®è©ŠéšãéããŠãèªèº«ã®æé·ãšãã£ãªã¢ã®åäžãç®æããŸãããã
6. ãŸãšã
ãã®èšäºãéããŠãCloudera Certified Developer for Apache Hadoop (CCDH)ã®è©Šéšã«é¢ããå€ãã®æ å ±ãæäŸããŸããããã®ã»ã¯ã·ã§ã³ã§ã¯ããã®å šäœã®ãŸãšããšããã®èªå®ãç®æãæ矩ããããŠä»åŸã®ãã£ãªã¢å±æã«ã€ããŠèå¯ããŸãã
6.1 Clouderaèªå®ãç®æãæ矩
Clouderaèªå®ã¯ãApache Hadoopã®å°é家ãšããŠã®æè¡ãç¥èã蚌æãããã®ã§ãããã®èªå®ãæã€ããšã§ãæ¥çå ã§ã®ä¿¡é Œæ§ãå°éæ§ãé«ãŸããå€ãã®äŒæ¥ããããžã§ã¯ãã§ã®æ±äººæ©äŒãå¢å ããå¯èœæ§ããããŸãã
ãŸãããã®èªå®ãååŸããéçšã§ãHadoopãšã³ã·ã¹ãã ã®æ·±ãç解ããå®éã®éçºç°å¢ã§ã®çµéšãç©ãããšãã§ããŸããããã¯ãä»åŸã®ãã£ãªã¢åœ¢æã«ãããŠéåžžã«äŸ¡å€ã®ããçµéšãšãªãã§ãããã
6.2 ä»åŸã®ãã£ãªã¢å±æ
Clouderaèªå®ãååŸããåŸã®ãã£ãªã¢å±æã¯ãéåžžã«æãããšèšããŸããããŒã¿ã®åãæ±ããåæãéèŠèŠãããçŸä»£ã«ãããŠãHadoopã®å°é家ã¯å€ãã®äŒæ¥ãçµç¹ã§æ±ããããŠããŸãã
ç¹ã«ã倧æITäŒæ¥ãéèæ©é¢ãããã«ã¯å ¬å ±æ©é¢ãªã©ãããŒã¿ã掻çšããŠæ°ãã䟡å€ãçã¿åºãããšããçµç¹ã§ã¯ãé«åºŠãªæè¡åãæã€Hadoopéçºè ã®éèŠãé«ãŸã£ãŠããŸãããã®ãããªèæ¯ãæã€äžã§ãClouderaèªå®ãååŸããããšã¯ããã£ãªã¢ã®å€§ããªã¹ãããã¢ãããšãªãã§ãããã
æåŸã«ãæè¡ã®é²åãå€åã¯åžžã«é²è¡ããŠããŸããClouderaèªå®ãååŸãããããšãã£ãŠãåŠã³ãæ¢ããããšãªããåžžã«ææ°ã®æè¡ããã¬ã³ããè¿œãç¶ãã姿å¢ããé·æçãªãã£ãªã¢ã®æåã«ç¹ãããšä¿¡ããŠããŸãã