Big data is one of the hottest topics in technology today, however it also is one of the most difficult to make sense of since the topic is vast. At the recent AT&T Developer Summit in Las Vegas, NV, the focus was on making sense of data in a world where virtually anything can pipe data to devices.
Fortunately big data doesn’t have to be horribly complex. As discussed by the panel, big data can be broken down into three key parts: volume, velocity and variety. The biggest goal is to find some value from all three. Unfortunately while there isn’t a shortage of data, companies are struggling to make sense of the information they have – one of the biggest challenges being knowing what information can be kept and what should be disposed of.
Big Data – A Simple Overview
For those unfamiliar with the term, Big Data simply is defined as a collection of information which is so large it cannot be processed by hand. Big data can apply to virtually any set of information imaginable – however the most common applications to date have been in advertising agencies (such as Google), retail stores (to predict inventory needs) and scientific fields (for research on different subjects).
Today however, with the advent of more powerful computers, companies and consumers are now saving troves of information – causing the demand for intelligent data processing solutions to surge in recent times.
Sources of Data
In 2014 and the years to follow, expect to see wearable devices become one of the biggest drivers of big data. Namely real time data will be the biggest niche in this area. Additionally customer data and transaction information also are major segments in the space.
In this case, it is vital for businesses to keep the 80/20 rule in mind. This means that 80% of effort should be focused on the data you have while 20% of effort is devoted to testing new and unknown information.
No Such Thing As An App
One of the best quotes from the presentation is that, “there is no such thing as an app.” Instead, developers today are providing a service based on information. The key to success is that developers and other professionals focus on asking the right questions and getting the right answers.
While making sense of information might seem near impossible, by clustering topics and focusing on similar information fields, you can easily make sense of big data.
Only Going to Get Bigger
Even if you don’t use data today, big data will eventually encompass all aspects of your life – literally. Thanks to Internet connections becoming mandatory for virtually all appliances both big and small, vendors will now need to process serious information (credit approvals, safety information, performance monitoring, etc.), and the mundane (gaming, recreation, etc.).
Although the poll below taken from the audience attendees at the big data session at the AT&T developer summit is not a scientifically sound sample of information, there is a key point to keep in mind. The most notable point is how 59 percent of participants stated that API’s are going to be a key focus on their development efforts. This means that rather than coding solutions from scratch and reinventing the wheel, APIs are allowing developers to plug in their code to established solutions as needed to build on top of existing systems.
In the AT&T hackathon for example, companies such as AT&T, Pebble, Qualcomm, and many other vendors provided attendees with innovative platforms which allowed them to code innovative solutions in only 24 hours.
Online storage startup Bitcasa has graduated from private beta and released Mac and iOS versions to compliment their current Windows, Windows RT, Linux, Android, and Google Chrome versions currently available on the market. Although the online storage market is already saturated with providers promising astronomically large amounts of storage space, Bitcasa stands apart from the crowd by promising users “infinite” amounts of storage for a nominal fee. As of the writing of this article, Bitcasa offers a free tier which includes 10 GB of storage. For only $10/month or $99/year ($69 for the first year) users are provided with infinite online storage.
Rather than just storing files, Bitcasa also promises infinite file versioning for their paid clients – a feature many of Bitcasa’s competitors lack. Aside from being office document friendly, Bitcasa is perfect for storing music, videos, and other multimedia files in the cloud because through direct access to files, users can literally enjoy their files without needing to download the data to their computer and then open it locally.
Although these features sound promising, as with all good things, there is a catch to this miracle technology. The ultimate pitfall of Bitcasa lies in its use of questionable encryption methods to save on storage costs. While many online storage providers provide full end-to-end encryption from the user to the server, Bitcasa uses convergent encryption. This means that although data is encrypted before being sent to the server, when it arrives, Bitcasa compares the encrypted artifacts to everything on their network.
After running a check, Bitcasa will then discard any duplicate bits while saving new data. Although Bitcasa claims this is a valid way to maximize the efficiency of their storage, the processgreatly defeats the purpose of encryption as it eliminates the anonymity and privacy protections provided by traditional encryption.
Bottom line – if a person owns the exact same file as you (say a music file purchased off iTunes), upon uploading, it is possible to see the unique signature for the file and then scan other users databases to see if they have the same file as you.
For a more technical explanation, the members of StackExchange discuss this aspect in detail on their Cryptology and Security forums. In September of 2011, Bitcasa CEO Tony Guada went on the record to address the encryption issues surrounding Bitcasa.
Is Bitcasa really worth the cost? The answer ultimately depends on what you plan to use the infinite storage for. If you plan to use it for office files, personal data, and other items which are not intended for other peoples eyes, then Bitcasa is not your solution. On the other hand, if you’re looking to store your multimedia files for streaming to your mobile devices and also to free up local hard drive space, then this might be the right option for you. Keep in mind that despite providing infinite storage, Bitcasa is not an ideal option for piracy related activities due to the previously mentioned encryption issues mentioned earlier.
Update – October 30, 2013: Bitcasa has updated their website to provide information on convergent encryption.
In today’s day and age, Big Data is one of the hottest terms on the market but what does it mean? Big data often is lightning rod because of the fact it is synonymous with the demise of privacy on the internet today. A shocking fact discussed in the opening portion of the book is that by the year 2025, the amount of data on the internet will exceed the brain capacity of everyone on the planet. Additionally the pace of which users are sharing data on the internet is occurring at a stunning pace – 90% of all new data on the internet was generated between 2009 to 2011.
How is this data used? First of all it is important to note anything on the internet is persistent and the data stored by computers is no longer limited to what you type into Google. As illustrated by the fact the Library of Congress has been archiving all tweets on Twitter since 1996 to the present. Genetic information usage in hiring practices has come up in the spotlight recently as a significant legal issue which does not have a solid boundary; while “Minority Report” style policing has been implemented in some metropolitan areas allowing police to predict criminal behavior from location, social network, and browsing data. Most shocking of all, metadata pulled from photos of individuals snapped within a “dating” application provided visitors with access to: criminal records, browsing history, or a site of dating reviews of individual people.
Behind all the computers processing this information are humans, which unfortunately are subject to making mistakes. Big data is a whole new area which is evolving much faster than traditional laws can keep up with. Unfortunately without regulations to guide corporations, business executives often have to guess which actions will balance effectiveness with privacy.
“Ethics of Big Data” solves this dilemma by providing a well thought out narrative which guides the user through a series of steps to generate a big data policy tailored around their companies specific needs. Rather than use the tone of a “teacher” by dictating how to implement a policy, Davis and Patterson guide the reader through a series of questions along with case studies to provide context, which lay the groundwork for the reader to develop their own policy specifically for their company. Additionally, as the book is written with a minimal amount of jargon, it is readable by all parties in a corporation ranging from management to the technical teams.
Overall, “Ethics of Big Data” is a must read for any company which has an online prescience, and at only 82 pages it is concise and to the point. The best aspect of this book is that regardless of your industry, company size and technological background, “Ethics of Big Data” provides the reader with a strong foundation to better understand what is one of today’s hottest industry topics.
“Ethics of Big Data” can be found on the O’Reilly online catalog
For business owners, the information age has been a bittersweet mix of increased productivity while also balancing ways to preserve mission critical data and files. Despite an increasing reliance on technology, many companies lack a sufficient disaster recovery plan when it comes to their systems.
Although CD/DVD’s, flash drives, external drives, and even tape devices have long been viewed as sufficient for backup, the all require user interaction for even routine tasks, having limited capacities, being prone to physical damage, and the issue of storing those backups in a secure manner. In addition, the crucial component missing from the previously mentioned systems is off-site redundancy, meaning that most backups are stored in the same location as the systems vulnerable to damage.
Today, many companies and individuals are turning to online backups as a solid addition to add to their disaster preparedness arsenal because storage is very affordable and the data centers powering the internet are being held to very stringent physical and digital security standards, essentially making them the Fort Knox of computer storage; a level of security most businesses can only dream of conforming to.
A pioneer for online backup in the small and mid-sized business sector (SMB) is Carbonite. Founded in 2005 with an original focus of providing “set it and forget it” backup services for the average consumer (currently priced at $59/year per computer), in June of this year, Carbonite launched a version of their service for SMB’s called Carbonite Business. The service is offered in Business and Business Premier Plans with 250GB of storage and 500GB of storage priced at $229 and $599 per year respectively.
The small business offering is ideal for many business owners in that unlike companies which charge “per user” fees, Carbonite Business lets you back up as many computers as desired as long as you have enough storage in your plan. The Premier plan includes the ability to backup Windows Server systems from version 2003 onwards, plus support for Microsoft SQL and Exchange backup files. Carbonite does not support backing up applications and operating system files.
In addition to the backup services, both plans include applications for the Android, Blackberry, and iPhone to ensure that you have secure access to your files no matter where you are.
Despite the extra capabilities in Carbonite Business, the program is as simple to use as the consumer edition, making it a breeze to configure your backups and to ensure that everything is on the up and up. Additionally, Carbonite has US-based technical support always ready from 8 a.m. to midnight several days per week, so assistance is never too far away.
In an interview with Pete Lamson, Senior Vice President and General Manager of the Small Business division of Carbonite, Lamson provided some validation of Carbonite’s service by pointing out that since its founding in 2005, the service has drawn over one million customers and that the service has backed up over 180 billion files and restored over 7.2 billion customer files. Regarding security, Lamson discussed how when Carbonite is used to backup data, they use bank grade encryption when the files are being transferred, and when it comes to storage, their data centers are fully compliant with not just banking standards, but also HIPAA and other rigorous standards which are required of mission critical centers.
When asked about the ideal market for their small business offering, Lamson said that they specifically designed it for companies with around three to twenty persons since their primary focus is providing affordable and reliable backup solutions to the small business market.
Going back to the fundamentals of the backup system, in addition to backing up your files as changes are made (or at a specified time), Carbonite Business also enables versioning with your files such that if you made an accidental change to a file, a prior version can be restored.
Despite the many advantages of Carbonite over non-online methods, one key point mentioned on the Carbonite website is that the service does not provide archival of data beyond 30 days, meaning that if a file is deleted from your system, it will be removed from Carbonite after 30 days. In addition, the real time backup capabilities mean that if a file is changed without you knowing, the file will be difficult to recover in the long run. The best way to combat this problem is to use Carbonite in conjunction with a secondary backup system (such as using imaging software such as Norton Ghost or Acronis True Image), although if you have very demanding backup needs, there is another provider which is worth mentioning.
Backup My Info! (BUMI) like Carbonite offers online backup, but in addition they offer permanent data archival, proactive monitoring of backups and storage, additional server technology support (Oracle, SQL, Windows, and others), and enhanced data transfer capabilities when it comes to the pipe between your company and the BMI data centers. The additional capabilities come at a premium as BMI charges at least $150/month for its services.
In an interview with Jennifer Walzer, President of Backup My Info, Walzer explained that BUMI’s service is specifically meant for companies with more rigorous needs and that their higher price allows them to offer enhanced services. She went on to discuss how BMI is more ideal for enterprises, especially which in finance, medical, and other data intensive environments as those industries typically need high performance solutions.
When asked if BMI had any plans to offer a more affordable solution, she said BMI has no plans to offer a lower priced solution as BUMI is keeping a focus on high performance backup which is difficult to adapt on a large scale without hiking costs.
Regardless of the methods you use to store data, if you have a computer it’s always important to have at least two forms of backup for each of your systems because as the saying goes, “You never should keep all your eggs in one basket.” For simplicity, however, it is ideal to have at least one online service and one offline service such that you can have some archival capabilities along with the security of having a backup in a highly secure location.
This article was originally written for SmallBizTechnnology.com – Original Post
Note: The print version of this article which contains the (somewhat NSFW) cartoon which inspired this article can be found here
Just as there are many ways to bang, there are many ways to bump. For reference though, the comic illustrating “bumping” is a pretty extreme illustration. It was the best one I could find on the Internet for purposes of keeping this column at an R level since NC-17 is too narrow.
Now going back to the technology. I’m talking about bumping phones to exchange information, be it contact information such as a business card or address book entry, photos, files, and even money via Pay-Pal, who officially created an application to use their system securely. The application is known as Bump (bu.mp).
Bumping, is intended as simulating a fist pound while holding your phones. Through the use of the many sensors within the Android and iPhone, the application lets you exchange data without actually needing to have the phones contact each other. Unlike Bluetooth and the now obsolete Inferred standards, which also do not require physical device contact, Bumping is much more flexible and easier and more enjoyable (puns not intended).
Although the platform is fairly new, Bump sports an impressive array of features from the ability to connect your Facebook, LinkedIn, MySpace and numerous other social networking accounts to compare contacts/friends between yourself and people whom you may bump.
As mentioned earlier, PayPal has integrated Bump into their official iPhone and Android applications, which allows you to lend money with friends or contacts without having to lug a large silver briefcase around with you all day. Now you have the option of carrying your phone, or being fancy and buying a silver hard case for your phone.
While overall Bump has a great concept, the technology is limited to matching phones meaning an iPhone must bump an iPhone rather than opposites like an iPhone bumping an Android. iPad users can also use the iPhone version of Bump and an iPad version is in the works.
Article originally published for The Statesma