Getting Onboard with HTTPS, Big Data Tech and The Search for Talent in Data Analytics – CF006

This week we talk news and implementation methodologies for HTTPS, take a first look at some of the newer big data top level projects at apache, and even get a bit philosophical talking about the search for IT and data analytics talent in the USA. Joined this week by a new guest and future co-host, Ashton Webster, and undergraduate in the Cybersecurity Honors College (ACES) at the University of Maryland, we have an hour of jam packed conversation that spans multiple topics in cybersecurity and big data.

Cyber Frontiers is all about Exploring Cyber security, Big Data, and the Technologies Shaping the Future Through an Academic Perspective!   Christian Johnson, a student at the University of Maryland will bring fresh and relevant topics to the show based on the current work he does.

Support the Average Guy Tech Scholarship Fund: https://www.patreon.com/theaverageguy

WANT TO SUBSCRIBE? We now have Video Large / Small and Video iTunes options at http://theAverageGuy.tv/subscribe

You can contact us via email at jim@theaverageguy.tv or call in your questions or comments to be played on the show at (402) 478-8450

HTTPS/SSL OVERVIEW

I want to get a message to my friend (HTML), but I realize that there is someone intercepting our messages.  How can I guarantee to my friend that there wasn’t someone in between (MITM)? By creating a message authentication code: if the message changes at all, the code changes.  And by the way, it would be nice to have encryption to hide the message, and compression to make it smaller…

• HTTPS = HTTP with SSL (now TLS)

• Introduction to SSL, how it works

  • The TLS handshake: agree upon a method of exchanging messages.  At the very least, guarantee authentication (MAC), optionally agree upon encryption and compression

  • prevents man in the middle attacks, beast attacks, re-negotiation etc.

  • Interesting site on ssl encryption shows which websites are vulnerable to ssl attacks – https://www.trustworthyinternet.org/ssl-pulse/

  • Usage statistics for ssl certificates: http://trends.builtwith.com/ssl  You can also search for virtually any software in the search bar, here is openssl http://trends.builtwith.com/Server/OpenSSL

  • http://www.libressl.org/ not the sexiest website, but hey, they are busy fixing the earth’s problems of insecure ssl

Implementing SSL on the Cheap:

https://www.globalsign.com/ssl-information-center/types-of-ssl-certificate.html

• http://www.ssls.com/ – $25 for 5 years worth of SSL certificate. (domain validated)

  • Levels: Domain, Organization, Extended Validation

  • For the “average guy” – here’s why domain validation is the way to go.

• Google PageRank is now evaluating SSL into its algorithm – http://arstechnica.com/security/2014/08/in-major-shift-google-boosts-search-rankings-of-https-protected-sites/

• Post-Snowden world – SSL adoption pretty much tripled – http://thinkprogress.org/world/2014/05/17/3438919/more-people-turn-to-encryption-after-snowden-leaks/

• Configuration Tips:

  • http://www.whynopadlock.com/check.php – Finding insecure elements on the page

  • SNI – Multi-host environment https://wiki.apache.org/httpd/NameBasedSSLVHostsWithSNI

OVERVIEW OF BIG DATA & APACHE

Starting at the bottom: HDFS http://hadoop.apache.org/docs/r1.2.1/hdfs_design.html

YARN

Serves as a broker between application and file system.  Resource management and runs multiple applications that interact with hdfs http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html

Brainstorming thoughts (AW):

• Christian – Hadoop story time?

• Storm – http://storm.incubator.apache.org/

  • Streaming data (*real time*)

  • moving from tuple to batch (define batch and tuple)

  • Before even entering the database! 

  • Storm can store its final results in hdfs still, and even get streams from it https://github.com/ptgoetz/storm-hdfs

  • Why Researchers should care

    • Process the data as it comes in would significantly cut down on the sheer volume of data to post process (i.e. a few gigs a second as it comes in instead of terabytes at once after)

    • analysis and collection virtually simultaneous = efficient

• Why Enterprise should care: Real time data means real time decisions (or at least faster ones)

  • list of companies that use storm: https://storm.incubator.apache.org/documentation/Powered-By.html includes spotify, weather channel, webmd, and other big names!

  • Intrusion detection systems

• Spark – https://spark.apache.org/

  • evolution of mapreduce (disk -> in memory)

  • scala and python – word count in a line instead of a bunch of

  • Machine learning (anecdote about ben and his long processes)

  • Why researchers should care

  • Why enterprise should care: Cuts down on time spent processing

• Tez – DAGS instead of multiple map reduce programs http://hortonworks.com/hadoop/tez/

  • Using vertices as processing points and edges as data flow (similar to storm streams and bolts) http://hortonworks.com/blog/expressing-data-processing-in-apache-tez/

Future topics…

• Signature based vs “baseline” based malicious behavior detection – True positives, false positives, etc. and cost of mistakes

• datalakes versus data warehouses

Jim’s Twitter: http://twitter.com/#!/jcollison

Contact Christian: christian@theaverageguy.tv

Contact the show at jim@theaverageguy.tv

Find this and other great Podcasts from the Average Guy Network at http://theaverageguy.tv

Music courtesy of Ryan King. Check out the Die Hard Cafe band and other original works at:
http://diehardcafe.bandcamp.com/ /  http://cokehabitgo.tumblr.com/tagged/my-music

Some links may contain affiliate codes that benefit the Average Guy Podcast Network.