Logo
Sam Eldin
CV - Resume Tools 4 Sharing Architects 2 Show Big Data Presentation Android Training Java-Unix Code Templates Interviews QA & Code

Database Replacement Using XML

To make our presentation short and sweet, if anyone searches Google looking for top issues with databases and top issues with Hadoop, Google search pages will display the followings:

     Top Database Issues      Top Hadoop Issues
       1. Growing complexity in landscape        1. Dealing with data growth
       2. Limits on scalability        2. Generating insights in a timely manner
       3. Increasing data volumes        3. Recruiting and retaining big data talent
       4. Decentralized data management        4. Integrating disparate data sources
       5. Performance        5. Validating data
       6. Security        6. Securing big data
       7. Resource utilization        7. Organizational resistance
       8. High availability        8. Production
       9. SQL injections        9. Performance
              10. Admin
              11. Concurrency

Database Storage and Data Objects Issues

Database and Data Objects
Image #1
Looking at Image #1 and tracing data in each of the cloud tiers, the following are serious issues:

1. Unmanageable number of tables and fields
2. Database Content would have: Errors, Redundancies, Duplicate, Dated Data, Inconsistencies, Not Used Stored Procedures and Functions, Duplicate Indexes, .. etc
3. Remote, batch, backup and data replication (GoldenGate) are added performance issue and security
4. Security issues in every tier
5. No consistency nor uniformity
6. Data moving through the tiers is tier and developers' dependent - no consistency or possible standardization
7. Too many conversions
8. Error prone
9. No reusability
10. Refactoring and transparency are difficult to achieve
11. SQL Injection can be added to any of the tiers
12. Slow performance
13. Difficulties in logging and tracing
14. Difficulties in audit trailing
15. Difficulties in database management - out of control

Our Database Replacement Using XML and NAS:



Our Database REpacement
Image #1


We can state with confidence that Big Data issues are still persisting regardless of the use of the top of the line databases or Hadoop and all Hadoop's hip.

Our approach and answers to Big Data, databases and Hadoop issues are as follows:

1. Growing complexity: is handled by making data an independent intelligent service(s)
2. Growing complexity: is also done by standardizing the Data Access Objects (DAO) based on the business types and performance
3. Limits on scalability: is addressed by using virtualization to accommodate vertical and horizontal scalability and expandability
4. Increasing Data Volumes: is done using XML files to store data on file servers (local and remote)
5. Increasing Data Volumes: is done by building straight conversion from and to XML and DAO (DAO and IDAO runs in memory and XML is stored in files) plus using NAS as low cost file servers.
6. Decentralized Data Management: is addressed with the fact that data would become an independent intelligent services and with use of virtualization, there are no restrictions on data storage locations or remote access.
7. Performance: is handled by using algorithms and long integers to create short presentation of the actual data for processing speed and updates
8. Performance: is addressed by building straight conversion from and to XML and DAO. Both DAO and IDAO runs in memory as memory resident and XML is stored in files.
9. Performance: is also addressed by building virtual XML-DOA converters which would handle any load
10. Updates: will be done with stanching speed since we have created short presentation of data mention in addressing performance
11. Security: is handles by using encryption and compression to secure data storage and IDAO which handle security within the running applications
12. Security and High Availability: are done by using IDAO and DAO would be the only data objects running within any application as data services, which can be passed as function parameters
13. Resource Utilization: is done by building reusable Data Access Object (DAO) and Intelligent (IDAO) to run as data services
14. High Availability and Performance: are addressed with memory resident IDAO and DAO, virtual XML files servers and IDAP objects stored on NAS for both reusability and history.
15. Conversions: is done by creating standards data structures for data communication, parsing and conversion based on the business types
16. Production, Testing and Rollback: are done using virtual servers and IP addresses (on demand) which they aslo would help in rollback and recovery processes.
17. Integration: is done using reusable, independent, Intelligent IDAO as data services plus the use virtualization where virtual data services (virtual servers) would be created on demands.
18. Parallel Processing: is handled by independent IDAO running in parallel to handle big processes with communicating IDAO
19. Concurrency: is handled using virtual data services with target IDAO and DAO
20. SQL injections and SQL Code: are totally eliminated - No more SQL
21. ...: we have more options to present, but we do not want to overwhelm the readers


Our approach is revolutionary and doable with low cost of development and integration. Stakeholders need to look at our approach and take the inchoative by teaming up with us. As one team, we can build a pilot project perfecting our solution. We would be taking the first steps in building intelligent data services which address Bid Data, CRM, BI and end the use of databases for good.