Big Data Dependency Mapping
Your big data environment likely contains several newer technologies like Hadoop and a NoSQL database such as MongoDB. It probably also includes a hodgepodge of other technologies – some commercial and some open source, e.g. Storm for stream processing, Dremel for ad hoc querying, Gremlin for graph analysis, and perhaps SAP Hana if they’re your ERP provider.
Once a big data project gets underway, senior business leaders want it to move fast so they can start reaping the benefits. Your project team is probably working longer than normal hours to build, scale up, test, and deploy those big data services for your business, and that’s great. Big data is one area where IT can really demonstrate its impact on the business’ bottom line.
- The speed of big data projects often means the knowledge you are accumulating around how to build and manage big data services, including their dependencies and interconnections with application data sources, isn’t getting documented very well
- Once big data services are available, and the business falls in love with using them, making changes to the supporting infrastructure could break them – putting your IT organization in a highly-exposed, negative light
- Making changes in the application environments that feed your big data services can also produce unexpected ripple effects and big data service interruptions
- Big data projects typically center on sensitive customer information so understanding how regulations and policies govern your big data infrastructure is also critical to avoiding audit findings and extended remediation efforts
The real big data challenge is not just being able to move fast, but ensuring you can modify and adapt your big data services and supporting infrastructure while also ensuring a state of continuous compliance for the data you are using.
- Easily create dependency models with the information you already have
- Validate dependency mappings so they are trusted
- Capture tribal knowledge so they are accurate and complete
- Provide shared perspective and personalized views
Fully understand relationships and
dependencies for big data environments
(including policy compliance requirements)