Calculating number π by throwing darts: digitally in SAP HANA

In my previous blog I promised exploration of Big Data on SAP HANA, express edition. But remember, the Big Data is not only about the Volume, but about Variety (of data types) as well. And this is the route I chose first to look at the fun stuff you can do with spatial data processing in SAP HANA.

Ever since I enjoyed “Calculating Pi with Darts” video from Physics Girl and Veritasium [which you should watch too!] I have thought about repeating it. The world is going digital, so obviously I meant using SAP HANA for that. I know I should have done it during the PI Day (3/14/16, or on the 14th of March 2016), but better later than never!

Calculating pi number with darts is one of the Monte Carlo methods of getting its approximate value. Accordingly to Wikipedia “[…] method for computing π is to draw a circle inscribed in a square, and randomly place dots in the square. The ratio of dots inside the circle to the total number of dots will approximately equal π/4”

Looked like SAP HANA’s spatial capabilities would fit perfectly for that. If you are not familiar with spatial processing I prepared four introductory tutorials that should not take more than 20 minutes for your to complete and understand all basic concepts needed to follow the rest of the blog. And if you do not have SAP HANA Express yet, then it is 10 minutes to get it. Alternatively you can use as well SAP HANA MDC instance in your HCP Trial account as we are still not talking about huge volumes of data here.

1. Points: http://www.sap.com/developer/tutorials/hana-spatial-intro1-point.html
2. Lines and strings: http://www.sap.com/developer/tutorials/hana-spatial-intro2-string.html
3. Areas and polygons: http://www.sap.com/developer/tutorials/hana-spatial-intro3-polygon.html
4. Spatial columns in tables: http://www.sap.com/developer/tutorials/hana-spatial-intro4-columns.html

Virtual dart hits are points with random X and Y coordinates (objects ST_Point). The dartboard is a disk (ST_Buffer() around a ring’s center point). And then calculation of the average of hits within an area of the disk (ST_Within() method).

First I need a table with a spatial column, which will store coordinates of my digital hits, plus a procedure to populate this table with required number of attempts.

CREATE SCHEMA "TESTSGEO";
SET SCHEMA "TESTSGEO";

--DROP TABLE "TESTSGEO"."SPATIAL_CALCPI";
CREATE COLUMN TABLE SPATIAL_CALCPI(
	POINT ST_POINT
);

--DROP PROCEDURE "TESTSGEO"."COLLECT_HITS";
CREATE PROCEDURE collect_hits (IN attempts INT)
 LANGUAGE SQLSCRIPT AS
 iter INTEGER;
 BEGIN
    iter := 1; 
    WHILE iter<=attempts DO
        INSERT INTO "TESTSGEO"."SPATIAL_CALCPI" VALUES (new st_point(RAND(), RAND()));
        iter := iter+1;
    END WHILE;
    MERGE DELTA OF "TESTSGEO"."SPATIAL_CALCPI";
 END;

Now let’s check the result of throwing 2000 virtual darts and what the PI number approximation will be!

--TRUNCATE TABLE "TESTSGEO"."SPATIAL_CALCPI"; 
CALL "TESTSGEO"."COLLECT_HITS"(ATTEMPTS => 2000);

--Check the results of throwing: coordinates and if hit dartboard
SELECT
  POINT.ST_asWKT(), 
  POINT.ST_Within(NEW ST_Point(0.5,0.5).ST_Buffer(0.5)) as IN_CIRCLE 
FROM "TESTSGEO"."SPATIAL_CALCPI";

--Calculating PI using Monte Carlo formula
SELECT
  4*AVG(POINT.ST_Within(NEW ST_Point(0.5,0.5).ST_Buffer(0.5))) as PI 
FROM "TESTSGEO"."SPATIAL_CALCPI";

Results I got in my system were between 3.11 and 3.21. Well, very rough approximation of number π 🙂

Let’s visualize the results by generating SVG with a dartboard and all generated hits.

SELECT
  ST_UnionAggr(POINT).ST_Union(NEW ST_CircularString('CIRCULARSTRING(0 0.5, 1 0.5, 0 0.5)')).ST_asSVG() AS DARTBOARD 
FROM "TESTSGEO"."SPATIAL_CALCPI";

I did a minor modification of the SVG to have a circle in red.

Then I tried 50000 attempts, but the result was 3.1168. So, no much improvement over previous attempts.

PS. Obviously using below SAP HANA spatial method calculating a circle’s circumference when diameter is 1 would be much faster and precise way to get the pi. But – hey! – it would take away all the fun of throwing digital darts 😉

SELECT 
  NEW ST_CircularString ('CircularString (0 0.5, 0 1.5, 0 0.5)').ST_Length() as PI 
FROM DUMMY;

--Result is PI 3.141592653589793

---

Please let me know what pi numbers you got by throwing digital darts in your SAP HANA instances.

PS. Republished from my blog https://blogs.sap.com/2016/12/14/calculating-number-%CF%80-by-throwing-darts-digitally-in-sap-hana/

Great Great experience at Mobile Mobile conference

I just returned from Kraków, where I had a pleasure to attend Mobile Mobile Conference. Well well done.

What did analytics guy focused mostly on Big Data and Fast Data (and Deep Data and Long Data and You-name-it Data) do at the melting pot of mobile developers and designers from around the world? Well, I’ve been following what’s going on in the mobile world for quite some time. The development of mobile technologies drives requirements for real-time data management, like this delivered by SAP HANA. Interestingly enough recently one student shared with me that his professor had stated “There is no business application with sub-seconds response requirements”. Seems this professor has not used any mobile device. Otherwise he would know that mobile apps reduced expectations for response times to a few seconds. It is not your granpa¹’s desktop PC anymore that you leave at the desk to process a report while you’re enjoying infamous “cup of coffee”.

The mobile development is currently dominated by consumer market, “consumerization of IT” in enterprises is driven by exactly the same force. Except of may be 2 Ps of marketing – Price and Promotion, – all other aspects of mobile apps business remain the same for both consumer and business markets. From that perspective the majority of sessions at MM Conf should have satisfied both camps of mobile professionals. The mix of topics and presenters was excellent to cover the area comprehensively yet without overlaps:

• Analysis of Emerging Technologies: smartwatches, connected cars, perceptual computers (Sacha Pallenberg)
• How accessibility enables Digital Inclusion (Robin Christopherson)
• The importance of user-centered approach, reminding me Design Thinking (Jeremy Olson)
• Cold shower for those thinking “how to get rich quickly” (Pratik Patel)
• Surviving your haters (Zach Holman)
• Devices’ sensors – the thing often overlooked in biz apps (Brian Suda)
• Internet of Things (Sam Soffes)
• Development for Emerging Markets (Patrick Leddy)

These laser-focused sessions were nicely enriched by

• general overviews (Aitor Garcia Rey and Steven Hoober)
• analysis of mobile development management in Spotify (Patrick Broman) and Github (Danny Greg)
• technical talks (Matt Thompson and Robert Virkus)

Small comment is that most of speakers were from the US, and not all cultural and market references were applicable to Europe. I hardly know any developer in Eastern Europe coding over a coffee in Starbucks… On the other hand, it was a unique chance for European developers to understand American way by listening and talking to presenters. And without need to spend $2000 for a trip over Atlantic.

To discuss the merit side of the conference and not to mention the organization side of the event is like to say the half of story in this case. Although it was the first edition of the conference this “small event” (as described at the conference’s webpage) gathered more than 180 participants from 16 countries. Not bad at all for the first time!

The secret sauce of this success was the team from Applicake. Guys reapplied same principles as at their already flagship Railsberry event. And if I can share the secret it was the treat of participant in every aspect: accompany informative (yet comfortably short!) sessions with long-enough breaks (for networking), nerdy location (Museum of Municipal Engineering in this case), yummy food (and not forgetting vegans), chilling zone (with comfy seating), fun activities (enriched with Spotify on site), insane gadgets (nerds underwear, anyone?). Have I missed anything on this list? Yup… Sunny weather. Well well done, Mobile Mobile conference!

The conference

—

¹ Actually I must categorize myself as one of these “granpas”. During a discussion with one of recently-out-of-college developers I mentioned that it’s always nice to have a game corner with Ataris and Commodores at technical conferences. He looked at me and asked “What is atari?”

Days 3 and 4 of ASUG SBOUC’12: More Education, Predictive Analysis and … see you next year

I am back home in Wrocław after ASUG SAP BusinessObjects User Conference (SBOUC) in Orlando, USA. Three and a half intensive days, yet I still feel like there was not enough time to discuss everything with everyone. But the first SAP InnoJam is just around the corner, so the time to pack and see folks again will come soon.

For the moment, let me go back to SBOUC to share highlights of the last two days.

More about Networking

Spammers attack at #SBOUC

In my previous post, I mentioned about the importance of face to face interactions. Social networking tools – no matter if we like them or not – play their important role too.  When during his Thursday’s keynote Don Tapscott asked how many in the audience are using Twitter, about one third raised their hands.

I am not an addicted fan of Twitter, but it became a handy tool for me. I don’t like the one player dominance on the market, I wish App.net, Path and alike good luck. Just remember “A fool with a tool is still a fool”.

And remember there are different kinds of “smarts” too. There was a moment, when spammers found that #SBOUC is trending on Twitter, and started their attack. First time I saw something like this (see picture). New tweets and handlers were coming faster than you were able to get what’s going on.

My session on SAP HANA

My session was obviously on the topic of SAP HANA. It was an updated version of the last year’s session, but now with the focus on where and how you can learn more about in-memory data management and where you can practice to gain hands-on experience.

If SlideShare content is not properly shown as embedded above, you can review it directly here.

Surprising that so many people want to learn SAP HANA and are asking for access to the software, yet so few are familiar with two SAP HANA Developer Center offers:

1. 30-days Test&Evaluation access to pre-configured virtual desktop hosted by CloudShare. The benefit of that option is that the desktop has as well popular and new SAP BusinessObjects BI tools: Explorer, Analysis for MS Office, Visual Intelligence.
2. Free developer edition of SAP HANA database hosted in the cloud. For the moment the only choice is Amazon WebServices.

Our Developer Experience team as SAP is responsible for Developer Center. One of our new projects is to add BusinessObjects to the family of DevCenters. If you have any suggestions or comments on BObj, HANA, Sybase or any other SAP technology on the Developer Center – please let me know in comments, via twitter @Sygyzmundovych, or just by sending old good e-mail to my SAP address.

SAP Analytics and Hadoop

It became usual to hear (mostly from the same people) that SAP’s innovations focused only on SAP HANA. Well, there were very good sessions during SBOUC showing integration of SAP Analytics products with Hadoop too. Below are some captures with session numbers, so you can download full sessions from ASUG Online yourself.

Reporting on top of a universe sourcing from Hadoop (session 1210 “SAP BusinessObjects BI 4.0 FP3 on Apache Hadoop Hive”)

SAP DataServices and Hadoop (session 202 “Another Buzz Word – Hadoop! Or is That Something a Regular Person Can Use?”)

Text Processing by SAP HANA and Hadoop (session 211 “Unstructured Data: Taming the Textual Tsunami with SAP HANA & Hadoop”)

New SAP BusinessObjects BI tools

In my previous post I mentioned new SAP BusinessObjects BI product called Visual Intelligence. Sessions around two other new products got no less interest:

SAP Predictive Analytics:

SAP Predictive Analysis positioned in the context (session 805 “Demonstration of SAP BusinessObjects Predictive Analysis 1.0 and Its Consumption from SAP BI Clients”)

SAP BusinessObjects Design Studio (aka SAP Zen):

Building BI app with SAP BO Design Studio (session 109 “SAP ZEN – BI Applications and Dashboard Designer”)

Custom Development with SAP BusinessObjects

Another topic interesting for me was the custom development with SAP BusinessObjects. Unfortunately not many people are aware of SAP BO SDKs and APIs, and even at SBOUC there were only two sessions on this topic. I hope to promote and to see more next year.

Use of SAP BusinessObjects SDK in BP (Session 501 “Free SDK Utilities to Help Manage Your Business Objects”)

Best Practices (Session 1307 “Introduction to the SAP BusinessObjects BI 4.0 RESTful and Crystal Reports JavaScript Viewer SDKs”)

SAP Analytics Forum

Unfortunately, because of my flight schedule I attended only one session during the SAP Analytics Forum at the day 4 of the conference. The session was Deloitte’s internal implementation of reporting using Crystal Reports and SAP HANA. Nothing speaks better than an example.

Deloitte presentation of using Crystal Reports with SAP HANA (click to enlarge)

See you next year…

… at ASUG SAP BusinessObjects User Conference 2013 in Anaheim, California.

In the meantime…

… if you feel that my posts haven’t covered the conference enough, please have a look at other blogs:

• “SAP Analytics Forum at User Conference Did Not Disappoint” by Avery Horzewski at SAP Blogs
• “SAP Announces EPM in the Cloud and Free Trial of Visual Intelligence” by Cindi Howson from BI Scorecard
• “SBOUC 2012 Wrap-Up: BusinessObjects Passion on Display” by Thomas Wailgum from ASUG News
• “At this year’s ASUG SAP BusinessObjects User Conference, BI4 loses its luster” by Ryan Goodman on his blog
• “SAP Business Objects User Conference – Day 1 + 2” by Dave Rathbun on his blog

Still not enough? Then you need to experience and describe it yourself 🙂

Days 1 and 2 of ASUG SBOUC’12: Education, Visualization and … Wars

The second day of ASUG SAP BusinessObjects User Conference (SBOUC) in Florida, USA came to the end. Two more days ahead.

Networking

I am trying to get full advantage of what’s most important on such events: education and networking. It’s ok to “network” on LinkedIn, Xing, SCN or whatever “platform” you are using, but still nothing replaces face to face discussions.

Education

My session at SBOUC 2012

When it comes to education, the face-to-face approach is preferable too, but online and offline possibilities are rich as well. If you are attending SBOUC this year, I obviously would love to see you tomorrow (Wednesday) at 1:30pm EST in N. Hemisphere A4 “Impact of In-Memory Technology and SAP HANA on Your Business, IT, and Career“. But even if you have something else already scheduled, or you are not in Orlando this week, ASUG did fantastic job this year again to have all session materials available online. You do not need to be conference participant, or even ASUG member to access and download materials. Enjoy! You will find the whole palette of sessions: from traditional (like BI strategy, Dashboards and Universe Migrations) to new and trending (like using BI4 with HANA, Sybase IQ, Hadoop Hive, or Advanced Data Visualization and Predictive Analytics). Lots to learn? That’s just a beginning, because to get a bit more structure into what you are learning, you can not go to “Learn BI” collection at http://www.sap.com/LearnBI to check official products tutorials for BusinessObjects BI 4.x, but as well for new products like SAP Visual Intelligence.

Visualization

I think new Visual Intelligence application (aka “VisI”) is the one getting most of the interest here at the conference:

SAP Visual Intelligence presentation during SBOUC’12

The good news is that besides reading Visual Intelligence tutorial, you can as well get 90-days trial from http://www.sap.com/tryvisualintelligence. Even more – you can join “Data Geek competition” and use data sets already prepared for you by SAP, or to connect to cloud-hosted SAP HANA instance: http://www.sap.com/datageek. Try it!

I personally used Visual Intelligence to do analysis of the survey results about our SAP HANA Developer Center, to see (what we already expected :-)):

Analyzing HANA DevCenter Survey using SAP Visual Intelligence (click to see bigger picture)

Developer Wars

The Developer Wars was a good end of the day: healthy mix of technology and fun, competition and community feeling. Here are teams presenting:

Team “Decision First” presents first

Guys from the second team ASTG(?) had lots of fun and used VisI too 🙂

The winner of this year’s Developer Wars was the team Optimal Optimizers

Team PieRates had many of the “SAP BusinessObjects Web Intelligence: The Comprehensive Guide” authors

The war is over. I am looking forward to the next year.

Keynote

Why did I miss to mention conference keynotes in my post? Luckily I do not need to redo, what has been perfectly done already by Timo Elliott on his blog here.

Closing remark

You can find live stream of fresh news from the conference on my Twitter: twitter.com/Sygyzmundovych, or you can search for #SBOUC hash tag.

Total Recall, or why and where I joined SAP

Douglas Quaid in Total Recall from 1990

There are moments in everyone’s life when you feel like it’s time for a change. Same happened to me. Ok, ok, may be not as dramatic, as it was for Douglas Quaid in Total Recall.

So, in July this year  I joined Technology Innovation & Developer Experience (DevEx) team at SAP. I remained located in Wrocław, Poland. Why?

Why SAP?

• I’ve been working with SAP as a customer and as a partner for last 12 years and I felt I have developed some kind of DNA. I am not good in chemistry, so will stop here 🙂
• The company is going through very interesting changes and challenges making themselves more open and expanding their portfolio to have a broader impact on IT not only in ERP space
• I wanted to be a part of these changes. Not just me: Chris Turner, CTO of Unilever, joined SAP as well. Great minds think alike 😉 [Disclaimer: Chris is now the boss … of my boss at SAP :-]

Why DevEx?

• There are many great teams doing interesting things in SAP, and I was in talks with quite few of them.
• But DevEx gave me what I had been looking for: getting back to technology, coding, developers, try/fail/learn etc – all the good stuff that drove me when I first saw a computer (in a book ;-)) and wrote my first BASIC code (on a paper :-D) back in 1986.
• DevEx is behind some of the most interesting projects and events in SAP, including:
  • Open Technology Innovation,
  • Developer Centers, including in-memory, mobile, cloud, HTML5 and growing,
  • Events like InnoJam, DemoJam and more.

Why Wrocław?

• Subjectively Wrocław is beautiful and one of the best cities to live, work and create; it reminds me San Francisco in many aspects.
  • Wrocław was the host of FIBA EuroBasket 2009, UEFA Euro 2012 and will host 2014 FIVB Men’s Volleyball World Championship.
  • The city has been selected as a European Capital of Culture for 2016. In 2017, Wrocław will host the World Games, a competition in 37 non-olympic sport disciplines.
•  The only drawback, there is no ocean :~(

Last word?

• Last, but not least: I wanted to stop travelling, and start spending more time with my family. That’s probably the reason why I am writing this post from Frankfurt airport while waiting for my plane to Orlando to bring me to ASUG SAP BusinessObjects User Conference. Therefore SBOUC will be the topic of my next post and my tweets during next few days.

SAPPHIRE and ASUG 2012 Orlando – Days 2 and 3

Finally it is my first week at home since mid May, and so it’s time to catch up on more news from the last days of the conferences. Hopefully you find below information interesting and relevant besides delay with making it published.

The thing that blew me out of my socks during the whole conference was a meeting with a group of start-ups, which are building their products using SAP HANA technology. This was the best show of taking the technology seriously. These guys are not big SIs or customers, who can make a wrong decision, fail, do finger-pointing, and then move on. For start-ups each decision is to live or die, and their founders are betting on the SAP HANA with their very own pocket-money. If you are interested to know more about innovative ideas coming from start-ups, or if you have your own idea and would like to know how to start working with SAP, have a look at “SAP Startup Focus Program“. It is real. I was contacted already by an Ukrainian entrepreneur with his own idea of a product using in-memory technology, and I was happy to refer him to this program.

The last day of the SAPPHIRE NOW was the “HANA day”. The whole keynote was focused on HANA, but it seemed to be built around the war with Oracle. The presentation of 100TB (of RAM) HANA system, which SAP assembled in their US data center, and the demonstration of fail-over capabilities were the answer to FUDs, which Oracle tried to disseminate earlier around SAP’s achievements in in-memory databases’ space.

No doubt the most exciting announcement for techies was the availability of free SAP HANA developers licenses!! It was a great achievement of collaboration between SAP and community influencers. You can go to SAP HANA Developers Center and get your HANA database development instance running in the public cloud few minutes later. Just to be clear: “free” is applied to SAP’s license, but not to the hosting cost. Make sure you read the post “Get your own SAP HANA DB server on Amazon Web Services” by Juergen Schmerder of SAP. Juergen is one of the heroes behind making SAP HANA available for the developers community.

SAP HANA Developers Center delivers a lot of materials to study already, but one more source is worth mention. At the same time the book “SAP HANA Essentials” by Jeff Word became available.

If you are interested to get this e-book for free, go to its registration web page and use a Voucher-Code 17C55987 to download.

As of today the book contains overview chapters, and will be updated with new and changing content over the next few months. [Disclaimer: I am one of reviewers working with Jeff on the content of this book]

Another interesting content, which was made available publicly during SAPPHIRE NOW, was “BW on SAP HANA Cookbook“. It discusses the aspects of planning, deployment, operations and modelling. Nevertheless, I always remember that one and only official documentation for SAP software is always help.sap.com, as SAP does not take responsibility for anything else published outside of the official help 😉

Quite interesting were as well two performance whitepapers:

1. “Analyzing SAP HANA’s Performance Test — SAP Changes the Total Value of Data Warehousing and Business Analytics” by Enterprise Applications Consulting.
2. “SAP and HP Breaking Analytic Performance Barriers with SAP HANA and HP AppSystems” by Enterprise Management Associates.

During the conference I had a chance to talk to Sharada Bose, who is a manager in HP’s performance group and who supervised the team running new SAP BW Enhanced Mixed Load Benchmark (BW-EML Benchmark). She said it had been an excellent collaboration between HP and SAP to deliver the results of this benchmark, and that the joint work was greatly appreciated by both companies. What was interesting about this new benchmark, Sharada added, is that the query part is completely ad-hoc and random. Most of the industry benchmarks, including the most known TPC-H, are based on a predefined set of queries. [Additional disclaimer: I am a day-time HP employee at the time of posting this blog, but observations and thoughts posted here are my own.]

I think information so far is wide and deep enough to take some time to read, digest and analyse. So, let me leave another interesting piece of information – new SAP BusinessObjects BI tool called Visual Analyzer – for a later blog.

SAPPHIRE and ASUG 2012 Orlando – Days 0 and 1

This week I am attending SAPPHIRE NOW and ASUG Annual two-in-one conference. There are many interesting developments going on in SAP in the areas of cloud, platforms, mobility, but my focus is broad BI/EIM/DM and this is what I am after at the conference. Co-location of SAP’s premier business conference and SAP users’ largest event gives a interesting – and sometimes polarized – view on the state of the business: you see cool HANA-powered mobile-steered 3-D demos on the show floor and you listen to the real customers’ far-from-perfect stories.

It’s been just two week since the end of my first SAP BusinessObjects BI 4.0 implementation project (my day life is a technology consultant), and the everyday life stories are closer to my heart than bright roadmaps. My first experience with SBO BI 4.0 was  polarized as well. On one hand the breadth of tools portfolio and user’s UI are way better than what SAP/BW could offer with its BEx applications. On the other hand, from the administration point I would expect much more from BI Platform’s CMC, as a single tool for administration, monitoring, and trouble-shooting. Yet all of these are a subject for a separate post. The point here is: with the realise  4.0 of SBO BI, I can easily say that it’s now at the stage where BW customers can and should start looking at the adoption of BusinessObjects BI tools as the front-end for BW data warehouse. Not completely there yet, but certainly incomparably better where it was three years ago. I ran into some discussions with fellow SAP Mentors over that statement. As I was complaining that we waited long 4 years for good BW/BO integration, my dear colleagues with pure BusinessObjects background, like Mico Yuk here, were complaining that SAP almost completely forgot about them and about innovations for the non-BW customers. What is your opinion?

The first ASUG education session I attended was by Jeff Duly on the topic of SBO Explorer, accelerated implementation by their BW shop. Besides few minor issues, the business response was overwhelmingly positive, just to confirm the statement that SAP’s decision to acquire mature BI tool set in 2007 was the right one (see also remarks above about the need to continue the integration and unification work).

The second ASUG session on my radar was the SAP HANA Ramp-up customer story. On the closer look, the story was rather about the benefits of BI implementation, as the customer moved from manual collection of data from four source systems and the development of reports in the MS Excel to the BI solution using SAP BusinessObjects’ WebI and Xcelsius as the front ends and SAP HANA database as the repository for the new data warehouse. What was extremely interesting in that session are lessons learned. I think they were extremely important to repeat them here:

1. Data. SAP HANA database by itself helps with the processing speed, but without proper data quality, you will get garbage out just faster. I blogged about this a year ago in “Critical Success Factors for SAP HANA implementations“
2. Ramp-up means “bleeding edge”. Experienced professionals know the meaning behind someone’s sentence “We learned a lot during this implementation” 😉
3. Realistic speed depends on complexity. In reality the performance of the query processing depends on many factors, major of which – when I/O bottleneck is eliminated – is the complexity of tables’ joins.
4. UI rendering time. SAP HANA database dramatically improves performance in data retrieval, but it is still only one of few steps on the way between user’s request click and the final display. Accordingly to presenters, the times to transfer data from HANA to the end-user machine and then get it displayed could go up to 20 seconds – leaving the impression of “long processing” even when the query processing in HANA had taken only 2-3 seconds.
5. Realistic time frame. They kicked off the project in May of 2011 with the plan of go-live in September 2011. But the acceleration of the data processing does not translate into the acceleration of project phases (although no doubt mean less frustration for everyone ;-), and the project went live 4 months later than expected.

The customer implemented the project with engagement of SAP Consulting and as I look around it is the case for almost every HANA implementation right now. If you are not a consulting arm in the one of the hardware vendors (HP, Hitachi, IBM etc) and if you are not one of the IT advisory companies (Deloitte, CapGemini etc) already working with the customer on the broad scale – it is difficult to get on the HANA implementation projects. It may be a cold shower for many smaller System Integrators (SIs). As HANA fewer spreads many of these SIs are establishing “HANA Centers of Expertise” and “HANA practices”, putting directors in place, making press announcements, but do not have projects… Here at the SAPPHIRE I met two of these “directors” already, who approached me asking how HANA implementation projects look like and what works and what not.

My first advise for them was: re-think what you want to do, analyse the broadening spectrum of HANA applications, and then try to focus. The thing is that SAP HANA world is huge and expanding and you cannot be in all places at all times. Especially if you are a small SI or a boutique firm.

My second advise was: SAP HANA itself is just an element of the broader SAP’s database portfolio. If you want to focus on the SAP database business, you need to make sure that besides HANA you can speak Sybase as well. If you want to focus on the SAP business applications powered by HANA (like Rapid Deployment Solutions – RDS), you need to speak the broad portfolio of SAP applications in the LoB or Industry area, which may not be powered by HANA today. If you want to focus on analytics, you need to speak broad portfolio of SAP BusinessObjects – BI and EIM – as well, and know how to build solution in SBO, which runs on HANA database, but as well on MS SQL Server, Sybase IQ, HP Vertica etc.

Going back to those who want to focus on the HANA as the database business with SAP, it is important that you separate hype from reality and understand the April 10th announcement of “SAP Real-Time Data Platform”.

HANA vs Exalytics: Where two fight the third one wins? Or is there a fight really?

The recent release of Oracle Exalytics brought third round of a verbal fight between SAP and Oracle and surrounding social (press, bloggers, forumers) comments. On a closer look you will find most of the arguments and comparisons pretty shallow and a based on rhetoric, then facts, be this from SAP side or Oracle side. Those keep discussion lively and catch the attention of many, but certainly do not please us – technologists, who are left-brained. One thing is for sure: megaplayers like SAP and now Oracle directed everyone’s eyes at the long existed, but previously niche, market of in-memory databases. Smaller, but established players, do not plan to waste 15 minutes of topic’s  fame, like Mike Allen, VP Product Management at Terracotta, cited in “Big Data Buyer’s Guide…” by Enterprise Apps Today:

At the end of the day SAP HANA and Oracle’s solutions are still databases – running on dedicated hardware that has to be sized to the problem in hand – that are accessed across a network. […] Terracotta BigMemory stores data right where it’s used, in the memory where the application runs. This makes it much faster.

Well, good luck to Teracottas, VoltDBs, Altibases etc, but for the moment let us move back to SAP HANA vs Oracle Exalytics debate. Indeed, at the moment SAP HANA is much closer in comparison to Oracle Exalytics, than to Oracle Exadata or to generic Oracle Database. I said “in comparison”, and not “in competition”, because to compete the products should stand against each other in customer bids, and not in vendors’ verbal arguments. So far I haven’t seen a single place where HANA and Exalytics stood against each other in the customer’s choice. I obviously do not see everything that is happening on the market, and therefore will greatly appreciate any additional facts in the comments to this post. On the surface there are many similarities between the two:

• both are using the word “in-memory” to describe the major characteristic of their major software component,
• both are sold only as software+hardware combo, although with Oracle Linux and Oracle Sun hardware with Intel Xeon chips in case of Exalytics, and with SuSE Linux and Xeon-based hardware from six (and counting) SAP’s hardware partners in case of HANA
• SAP’s initial major and most promoted use cases for HANA database were analytical data marts and planning applications, same (but only these) are use cases for Exalytics;
• both are Fast Data solutions, requiring to couple them with Big Data solutions – Exadata in case of Exalytics, or database from Sybase family in case of HANA.

So, what make them different and why we don’t see really a fight on the market between the two, although for sure both vendors are searching prominent example to announce their superiority and victory?

1. First and the foremost: the position of the product on the vendors priority list. HANA is the center of the SAP’s technology strategy: to-be the heart of all-things-SAP plus a platform with which SAP tries to attract ISVs. For Oracle Exalytics is just an answer to first technical use case released by SAP for HANA, i.e. high-performing analytics on data marts.
2. From the go-to-market, I see both as products to be up-sold or cross-sold, and therefore positioned in vendors’ own shops, rather than going to brand new industries and customers and clash there. There are certainly examples of HANA and Exalytics sells into new accounts, but I do not see these happening in volumes both vendors are after.
3. Exalytics has BI engines and tools pre-installed, while HANA doesn’t. Those who follow HANA’s history right from the beginning may remember that SAP initially wanted to include BusinessObjects BI tools run-times on HANA appliance as well, but finally did not. Instead today HANA database has lightweight application server (called XS Engine) built-in. As such Mark Hurd of Oracle was right in last Q3/2012 earnings call: “… one of the nice things about Exalytics is you plug it in and your existing Hyperion EPM system runs faster. Your existing Oracle BI runs faster. With HANA, one of the problems is you plug in HANA and then you start to write programs, because it’s not compatible with anything.” The thing though is that with HANA you can obviously write programs, but as well you can plug it into some of the SAP applications: ERP accelerators, SAP NetWeaver BW, SBO Analytic Applications.
4. And that’s another difference: Exalytics works only with Oracle software, and HANA’s strongest case is in interoperability with SAP products. HANA database will not work with Hyperion EPM (even if provocatively invited by SAP), and Exalytics cannot be used with SAP BW. Although interestingly now the use case for SAP BW puts SAP HANA database into the competition with Oracle Exadata, which has been certified for SAP NetWeaver installations.

BW 7.3 Orange (powered by SAP HANA) is just the beginning

In the podcast “Debating the Value of SAP HANA” mentioned in my previous post, we ran into the discussion if SAP NetWeaver BW 7.3 SP5 powered by SAP HANA (project “BW Orange”, or simply “BW-on-HANA”) is the killer app or not for SAP in-memory technology. My answer was “Yes”, but here I would like to go into the grades of shade beyond this binary question.

There are very few customers out there, who would not complain about some aspects of the performance of SAP/BW. I had been presenting sessions on BW performance at different conferences since 2008, and they always resonated with the audience. BW-on-HANA, which comes with the major promise of performance boost for both front-end (querying/planning) and data warehousing parts, will resonate with almost every BW customer. I see it in my day job as a consultant.

It is a no-brainer that reading all data at the speed of RAM, instead of disks, will resolve lots of I/O-heavy processing. You may share my memories of frustration, waiting for long-running BW operations – queries, DTPs, extractions, where the only proof that the work session is still alive was to go to transaction DB02, find your running SQL and then see slowly but steadily increasing numbers under “blocks read”. This should be a song of the past with BW running with HANA system. The advantages of in-memory data retrieval are obvious, and there should be no discussion about this aspect of BW Orange.

But I/O bottleneck is not the only reason for slow performance. Accordingly to SAP’s own statistics presented during SAP TechEd’10 USA in ERP about 20% of the processing is happening in the database layer, and the rest 80% is in application (ABAP) layer. Obviously in this case super fast database can resolve only 20% of performance issues. Althought the ratio 20/80 will be different for BW, as more heavy BW joins or massive inserts are done, the problem remains the same. Through the years BW (like most of the other ABAP-based systems) was built to run lots of the intensive processing in the application layer. Analytics (OLAP and Planning) Engine) being the prominent example. DSO activations, SID calculations, user exits are some more examples from the data warehousing part of BW.

Therefore BW Orange release introducing some applications optimizations, which are specific to SAP HANA and cannot be reused by BW running on any other database system. You may have heard about those by now:

1. HANA-optimized InfoCubes, i.e. different technical design of underlying database tables, and different querying technology,
2. HANA-optimized DSOs and the new activation process executed in the db layer,
3. New Planning Applications Kit, which allows execution of BW-IP standard functions within HANA db.

As you see the list of optimizations is not going through all BW components. Additionally, optimizations are not 100% applicable, i.e. in some cases tables layouts remain the same as on regular databases and processing is executed in ABAP layer, like in case when DSO is  supplied by RDA real-time data acquisition (see OSS Note 1665322 – Conversion for SAP HANA-optimized DataStore objects). In some other cases “Remodeling might be required to leverage Hana optimization when starting from an existing scenarios” (OSS Note 1637199 – Using the planning applications KIT).

The human brain and hands are not yet completely eliminated, and we all are just at the beginning of the extremely interesting journey.

Big Data and SAP HANA? Or Sybase IQ?

Like few more folks I think that there was some kind of misunderstanding in mixing Big Data and SAP HANA into one bag. We touched on this topic in the recent podcast “Debating the Value of SAP HANA”, but I would like to spend few more minutes here to explain my thoughts.

SAP HANA has been created with traditional SAP Business Suite and Business Warehouse (BW) customers in mind. How big is the biggest single SAP software installation in the world in terms of single-store data size? I do not know exactly. The times of the proud “Terabyte Club” are in the past. Four years ago it was loud about 60TB BW test SAP did. The biggest customer I worked with had 72TB database of BW data. So, I would assume that the biggest SAP instance is somewhere close to 120 TB. That’s still a lot of data not just to process, but as well to manage (think back-ups, system upgrades, copies, disaster recovery etc)… Besides current technical limitations – 8TB biggest certified hardware configuration and 2 billion records limit in a single table partition – SAP HANA is on the way to help SAP ERP and BW customers with those challenges. But those are not what the industry calls “Big Data”.

Here are main differences as I see them:

• Data sizes we are discussing with SAP HANA are in the ballpark of few terabytes, while Big Data currently is something in single digit petabytes. E.g. HP Vertica has 7 customers with a petabyte or more of user data each accordingly to Monash Research.
• Current focus of SAP HANA is structured data, while Big Data issues are generated by mostly unstructured data: web, scientific, machine-generated. Fair to mention though that SAP is working on Enterprise Search powered by HANA, as  Stefan Sigg, VP In-Memory Platform in SAP, told me during this TechEd Live interview.
• Currently Big Data processing is almost a synonym with a MapReduce software framework, where huge data sets are processed by a big cluster of rather cheap computers. On the other hand SAP in-memory technology requires “a small number of more powerful high-end [servers]” accordingly to Hasso Plattner’s “In-Memory Data Management: An Inflection Point for Enterprise Applications” book.
• Related to the point above is that in SAP HANA the promise is the real-time, where fact is available for analysis subseconds after occurrence. In Big Data algorithms processing is mostly batch based. My previous blog’s post became available in results of the Google Search and in Google Alert only 4 days after being posted – not quite real-time, huh?
• SAP HANA data analyses are most often paired with SAP BusinessObjects Explorer – modeless visual data search and exploration. Use of MapReduce libraries on top of Big Data requires advanced programming skills.

During SAPPHIRE’11 USAkeynote speech Hasso Plattner mentioned MapReduce as a road map feature for SAP HANA, but since then I haven’t gotten any specifics what it means. Instead silently announced Release 15.4 of Sybase IQ has introduced some features focused on analyses of Big Data in their original meaning. Is there a silent revolution in SAP going on the Sybase side, while all eyes are on the HANA product?