Treasure Data and Heroku

  • Published on

  • View

  • Download



<ul><li> 1. Treasure DataTreasure Data and Heroku Masahiro NakagawaHeroku Meetup #8 TreasureData + Waza Report!! Thu, 04 Apr 2013Friday, April 5, 13</li></ul> <p> 2. Who are you? Masahiro Nakagawa @repeatedly / Treasure Data, Inc. Senior Software Engineer, since 2012/11 Open Source projects D Programming Language MessagePack: D, Python, etc... Fluentd: Core, mongo, etc... etc...2Friday, April 5, 13 3. Introduction toTreasure DataFriday, April 5, 13 4. Company Overview Silicon Valley-based Company All Founders are Japanese Hironobu Yoshikawa Kazuki Ohta Sadayuki Furuhashi OSS Enthusiasts MessagePack, Fluentd, etc. 4Friday, April 5, 13 5. Investors Bill Tai Naren Gupta - Nexus Ventures, Director of Redhat, TIBCO Othman Laraki - Former VP Growth at Twitter James Lindenbaum, Adam Wiggins, Orion Henry - HerokuFounders Anand Babu Periasamy, Hitesh Chellani - Gluster Founders Yukihiro Matz Matsumoto - Creator of Ruby Dan Scheinman - Director of Arista Networks Jerry Yang - Founder of Yahoo! + 10 more people and.... 5Friday, April 5, 13 6. Treasure Data = Cloud + Big Data CloudBig Data-as-a-ServiceDatabase-as-a-service EnterpriseLightweight RDBMS TraditionalRDBMS Data WarehouseDB2On-Premise$34B $10Bmarket market1Bil entry Data VolumeOr 10TB 2012 Forrester Research, Inc. Reproduction Prohibited6Friday, April 5, 13 7. Why Cloud? Time is Money IdealCustomerExpectation ValueMaintainObsolete RedShift over time EMR AWS (or hosted Hadoops) Reality EC2Step-by-step manualintegrations(On-Premise) S3UpgradeHW/SW Selection, PoC, Deploy... TimeSign-up or PO 7Friday, April 5, 13 8. Full Stack Support for Big Data ReportingOur best-in-class architecture Data from almost any sourceand operations team ensure the can be securely and reliablyintegrity and availability of your uploaded using td-agent indata.streaming or batch mode.Our SQL, REST, JDBC, ODBCYou can store gigabytes toand command-line interfacespetabytes of data eciently andsupport all major query toolssecurely in our cloud-basedand approaches.columnar datastore. 8Friday, April 5, 13 9. 9 Product Data CollectionData Warehouse Data Analysis Web LogBI Tools App Log Streaming Upload REST Tableau, QlickView Open-Source JDBC / ODBC Log Collector &gt;60billion / monthExcel, etc. Sensor Columnar StorageSQL 2,000+ companies (HiveQL) (incl. LinkedIn, etc) +orHadoop Pig Bulk Loader MapReduce RDBMS DashboardCSV / TSVBulk UploadCRMMySQL,Parallel Upload Result push Custom App,250bil+ records PostgresRDBMS, FTP, etc.2mil+ jobs ERPOracle, etc.Value Proposition:Time-to-Answer 20bil+, 2 weeks, 2 weeks, 3 weeks, 3bil+, 3 weeks 2 weeks, UK/AustriaUS Japan Singapore US Multi-Tenant: Single Code for Everyone - no code modification, Improving the Platform Faster.Friday, April 5, 13 10. Customer Use CasesFriday, April 5, 13 11. 11 Our Customers Fortune Global 500 leaders and start-ups including:Friday, April 5, 13 12. Example in AdTech: MobFox 1. Europes largest independent mobile ad exchange. 2. 20 billion imps/month (circa Jan. 2013) 3. Serving ads for 15,000+ mobile apps (circa Jan. 2013) 4. Needed Big Data Analytics infrastructure ASAP.12Friday, April 5, 13 13. Two Weeks From Start to Finish!13Friday, April 5, 13 14. Global HuluFriday, April 5, 13 15. Before Hard to manage Hadoop Complicated data collectionFriday, April 5, 13 16. After No more Hadoop maintenance Versatile data collector, td-agentFriday, April 5, 13 17. Our UsageFriday, April 5, 13 18. 18Friday, April 5, 13 19. 19Friday, April 5, 13 20. Other usage Staging environment Internal testing application Proxy server for our used services20Friday, April 5, 13 21. Heroku integrationFriday, April 5, 13 22. data-is-sponsoring-heroku-waza-201322Friday, April 5, 13 23. Matz, April 5, 13 24. Heroku addons, April 5, 13 25. 25Friday, April 5, 13 26., April 5, 13 27. Using Heroku addon Setup td command Install via td-toolbelt or rubygems Setup td heroku plugin heroku plugins:install Add td gem to your Gemfile or STDOUT log collecting heroku td is now available for Treasure Data heroku td xxx: xxx is the same as td command 27Friday, April 5, 13 28. Just STDOUT Use STDOUT to collect event logs No need libraries log forward via Heroku syslog drain Format @[db_name.table_name] json_in_one_line Ruby:puts @[service.users] {"name":"D", "via":"Phobos"} 28Friday, April 5, 13 29. 29Friday, April 5, 13 30. Conclusion Treasure Data Cloud based Big-data analytics platform Provide Machete for Big data reporting Heroku and Treasure Data Treasure Data addon easy to integrate with your Heroku app STDOUT log collecting with Heroku syslog drain 30Friday, April 5, 13 31. Big Data for the Rest of | @TreasureDataFriday, April 5, 13</p>