Quantcast
Channel: OBIEE in IL
Viewing all 167 articles
Browse latest View live

OBIEE 11.1.1.7 and 11.1.1.9 without flash in HTML5 and PNG formats

$
0
0
At the moment using Flash Player is sometimes considered vulnerability. Some browser temporary disable it or threaten to do it, until the solution is found. I'll talk about Using OBIEE 11.1.1.7 and 11.1.1.9 without Flash. 



For 11.1.1.9 Andrew (BIsoftDiary.com) did a great job covering it in his post OBIEE: HTML5 charts instead of flash charts. He suggests to add the line <DefaultWebImageType>HTML5</DefaultWebImageType>  under views -> Charts in instanceconfig.xml. (See few more steps of service restarting and browser issue in his blog).
This option was described in 11.1.1.9 new features here.

All possible options of DefaulWebImageType are:
  • flash - In a browser that does not support the flash format, the image does not render. You should use the html5 value instead.
  • png (W3C Portable Network Graphics)
  • svg (W3C Scalable Vector Graphics) The svg value is not supported in this release, so flash is used if svg is specified.
  • html5 - In a browser that does not support the html5 format, the image renders in the flash format instead.
 Flash, png, and html5 images provide the greatest degree of interaction because they support mouse-over behaviors (such as popup data labels), navigation, and drilling. 

The only new feature in the 11.1.1.9 is the html5 option. 
For 11.1.1.7 we can use the png option with almost no problems. The png option supports drilling on graphs and most of the critical functionality, but can't do sliders.

In the example the Years column is at the Section level in the graph. I can mark the slider.


In HTML5 format it looks fine:
In 11.1.1.7 if we don't want the default flash, we are limited to png format that doesn't support sliders:


P.S.

If you are testing it, don't do the tests on the default Sample Sales dashboards.

BDD - Oracle BDD 1.1 (Big Data Discovery 1.1) is available

$
0
0
The second release of Oracle Big Data Discovery (BDD), 1.1 is available.
You can find it's main page here: https://www.oracle.com/bigdata/big-data-discovery/index.html. That includes data sheet, executive briefing and video. You can read there about Capabilities.
The OTN page is here.
The documentation of BDD 1.1 is here.
The YouTube channel is here.
The Learning library is here.
The download is from edelivery (search for Big Data Discovery).

To get a list of the new features in BDD 1.1, I used the Cumulative Feature Overview Tool described in https://blogs.oracle.com/proactivesupportEPM/entry/cfo_web_interface_bi_bdd


  • Hortonworks Data Platform (HDP): Oracle Big Data Discovery supports the two most widely-used Hadoop distributions; Hortonworks Data Platform and Cloudera Distribution for Hadoop.
  • Kerberos in Hadoop: Oracle Big Data Discovery runs on Hadoop clusters secured by Kerberos authentication, allowing secure analysis of big data across whole organizations. 
  • Spark on YARN: Oracle Big Data Discovery uses Spark on YARN, making data processing faster and more stable. 
  • Multiple versions of Cloudera Distribution for Hadoop (CDH): CDH support has been expanded to include multiple versions, so you can install on a wider range of systems and updated to incorporate improvements to the evolving Hadoop ecosystem. 
  • Options for initial data loading: A new self-service data load experience makes it even easier for business analysts to build innovative data combinations that bring together Hadoop and desktop data. 
  • Data set and project level security: Access to individual data sets and projects can be restricted for specific users and user groups, allowing Oracle Big Data Discovery to be rolled out to large groups of analysts securely. 
  • Update and reload data: Data sets can be updated from within Studio both manually and automatically, making it easy to keep projects and dashboards in synch with rapidly changing data. 
  • Big Data Discovery applications: A business-user friendly experience that supports a whole range of data operations for projects and applications: Projects let business users perform ad-hoc exploration and discovery using standard and advanced analytic techniques. Applications expose interactive analytic dashboards to a broad set of users and are set up to periodically receive new or refreshed data. 
  • You can publish and share transformation scripts, accelerating the process of data cleansing. 
  • Transform enhancements: Support for Geotagger, Whitelist, and Entity Extraction transformations on the Studio Transform page make it easier to derive structured data from messy Hadoop sources without writing code. A redesigned Transform Editor provides more help to business analysts who need to go beyond one-click transformations. Transform performance is improved, tightening the cycle of improving data quality in Transform and analyzing the results in Explore and Discover. 
  • Data visualizations are standardized and enhanced to allow easier visualization interpretation and interaction by business analysts and data scientists alike. A refreshed configuration experience makes the creation of data visualizations faster while also offering more control. 
  • A new Custom Visualization Component makes it easy for you to build and deploy interactive data visualizations using JavaScript and EQL. 
  • Enhanced upgrade experience allowing in-place upgrades to new Oracle Big Data Discovery releases. 
  • A new JDBC connector allows IT to securely set up connections to gold-standard enterprise databases. Business users can then add data sets originating in JDBC and use them along with data sets originating in Hadoop. 
  • The bdd-admin script includes a number of new capabilities that give administrators more options for managing Oracle Big Data Discovery. 
  • Language processing capabilities are expanded to support more than 80 languages. 
  • Log files contain new properties that give administrators more control over their behavior and the information they provide. 
  • The Enterprise Manager plug-in includes new metrics and operations that make it easier for administrators to monitor and manage Oracle Big Data Discovery clusters. 
The Oracle Big Data virtual machine 4.2.1 available here, now includes BDD 1.1

BDD 1.1 VM is availble as part of the Oracle Big Data VM

$
0
0
BDD 1.1 VM is available as part of the Oracle Oracle Big Data Lite Virtual Machine here, as part of the version 4.2.1.
Please note: This appliance is for testing and educational purposes only; it is unsupported and not to be used in production.  It includes software products that are optional on the Oracle Big Data Appliance (BDA), including Oracle NoSQL Database Enterprise Edition and Oracle Big Data Connectors (and now, BDD).

Oracle DB SE2 12.1.0.2.0 is available

$
0
0
Oracle DB Standard Edition 2 (SE2) 12.1.0.2.0 is available for download from Oracle here.The link to it's page is here.

At the moment it's available for the following platforms:
  • Microsoft Windows x64 (64-bit)
  • Linux x86-64
  • Oracle Solaris (SPARC systems, 64-bit)
  • Oracle Solaris (x86 systems, 64-bit)
  • HP-UX Itanium     
  • AIX (PPC64)
  • zLinux64
It is described as: 
Oracle Database Standard Edition 2 is an affordable, full-featured database built for unprecedented ease of use, power, and performance. From single-server environments for small business to highly distributed branch environments, Oracle Database Standard Edition 2 includes all the features necessary to build business applications.

  • Optimized for deployment in small enterprises, line-of-business departments, and distributed branch environments
  • Offers customers a container database architecture, making it easier to plug into the cloud
  • Includes rapid application development tools and supports a wide range of developer frameworks and data types including XML and JSON
  • Available on servers supporting a maximum of two sockets on all Oracle-supported operating systems, including Windows, Linux, and Unix
  • Provides complete upward compatibility, protecting your investment as your usage requirements grow

OBIEE - Line Breaks

$
0
0
Customer asked for line breaks in analysis. This was basically covered before for example here by obiee10grevisited.blogspot.com but I felt it needs so more examples.
(done in OBIEE 11.1.1.7.140527)

Lets create a simple Analysis with 2 column, on of them the string I want line breaks in. The mark for line breaks in OBIEE is <br>. So my formula is: 'Hello<br>people'

This is the formula:

To force <br> to work we will set the text to be HTML in Data format:


Now we see it's working:

You might want the line break to work in the header as well. In that case, select the “contains HTML Markup” in Formula (or in Column Properties->Column Format) :

I was very much surprised to see the export and printing options handling this well not only in HTML output, but also in PDF:



In case your data does not contain <br> you can always use the replace function.

For example:

Replace('Hello !! People','!! ','<br>')

Special characters in data should work as well:

Replace('Hello'|| CHAR(13) ||'People',CHAR(13),'<br>')
 
 This kind of columns might cause some problems in Graph legends.

OBIEE - Few options to compare data to previous month, Year or something...

$
0
0
I was asked to help with comparing data to previous period, month or year. There was another problem: No time dimension. I will describe here few options to do similar things.
(This post took me long time to finish. It started with 11.1.1.7 and was finished with 11.1.1.9 but should work with both)

Option 1: the correct option, use AGO function

This is the short and default answer. Create a date dimension. Make sure you have chronological key for at least date and month levels (date only might surprise you when comparing shorter and longer month) and use AGO function as described in the post: OBIEE and Time Series Calculations in analysis (Ago‎, Period Rolling‎, To Date‎).
For example:  AGO("Base Facts"."Revenue", 1)








The following options will be without time dimension. All done on analysis based on Sample Sales, with Product_Type, Revenue and some period (Year or Month or both).

 

Option 2: Calculating previous month using MSUM (relevant for non chronological as well)

The idea behind this option is the calculation: 
Sum(Last 2 Periods) = Previous Period + Current Period. As a result:

Previous Period = Sum(Last 2 periods) - Current Period

How do I calculate Sum(Last 2 Periods)? Using moving Sum. In our case: MSUM(measure,2)

Example 1: I have only Month and Revenue.
To calculate previous Month I add a column with formula MSUS(Revenue,2)-Revenue
Next I can do any calculation I want with the 2 columns.


Example 2: having Product Type, Month and Revenue 
When adding dimensions, things get a little complicated. I'll add "Product Type" column:
We have a problem with the first period. Instead of being zero or Null, it's the last value of the previous Product Type!

Since I'm lazy, I'll just add a case statement, for the first period of each product type, the value is NULL, else the previous calculation (just remember, first periods might be different for various Product Types).
First Period calculation: Min(Month by Product)
The case should be: CASE WHEN (Month=Min(Month by Product) then  NULL ELSE (MSUS(Revenue,2)-Revenue) END

Next I can do any calculation I want with the 2 columns.

By the way, some basic math using msum(Revenue,2):

Prev. period: msum(Revenue,2) - Revenue
diff between 2 periods: 2*Revenue - msum(Revenue,2) (the calculation: Revenue - prev. period)
% of change between periods: 100*(2- msum(Revenue,2)/Revenue) (the calculation: 100*diff/Revenue)


Option 3: Calculating same month of Previous Year

In many cases the data has strong seasonality, so it's logical not to compare and make calculations for the 2 month, but rather the same month of previous year.

To make things easier I used constant Years calculation. You can create a general analysis by replacing it with Presentation Variables or based on Current Year calculation:

Current Year:  YEAR(CURRENT_DATE)
Previous Year: YEAR(CURRENT_DATE) - 1

For this option to work I need the Month value to be the same for every year. In Sample Sales the month format is YYYY/MM, so I used the function RIGHT(Month, 2). Please note: in Sample Sales, both Year and Month columns are characters.

I filtered the data to be only Years 2009 and 2010.
Since I don't have a Year column. The initial data is sum of both years.


I'll add a column to calculate year 2010 Revenue only:  FILTER(Revenue USING (Year = '2010')) and one for Year 2009 FILTER(Revenue USING (Year = '2009'))

In case you have a presentation variable, named, for example, P_YEAR1 with the year 2010 selected, you should have the following formula: FILTER(Revenue USING (Year = '@{P_YEAR1}{2010}'))

Now we can remove the Revenue column and do any calculation we want with the 2 columns. (In the example bellow, I'm showing only fist 4 month of year and calculate the difference between the two years):


If you want a nicer column headers, but don't want them to be static... In case you use the analysis in a dashboard, you can use presentation variables as column names, such as: @{P_Year1}{2010} and use the same Variables as the analysis filter.


Option 4: Creating specific calculations using the $number calculated item (relevant for non chronological as well).

Assuming we want to compare specific values, lets say we want to compare 2 first month, 2 last month and last vs. first...
We can create a new calculated item and use $number to point to specific row / columns:
A useful and less known fact $-number would count from the end. 

 For example if we have 12 month in a pivot:

We can add calculated item that returns the difference between first 2 month: $2-$1
the last 2 month (if  I know they are 12 month)

Or if I don't know how many month they are ($-1 - $-2):


last month minus first month would be: $12-$1 (or $-1 - $1)...



Option 5: Using Filter to select specific periods (relevant for non chronological as well). Similar to option 3.

We can use filter function to return specific values.
For example we only want 2 first month of 2009 and the total revenue of 2009. I'll select the filter function:

Filter it by month:

select the condition:


and create the function:
the function is:
FILTER("Base Facts"."Revenue" USING ("Time"."Per Name Month" = '2009 / 02'))

We can replace the specific value (2009/02) with prompt.
Any calculation is now possible between the functions.

This will work if the aggregation level of the data is higher than the filter level. In our case a year:

Not if you add the month to analysis:





Option 6: Using Evaluate with the (Oracle) DB LAG function.


We can use database functions such as Lag and run them from Evaluate function in OBIEE. I leave it to you, if you are interested.

 

OBIEE BUNDLE PATCH 11.1.1.7.151020 AKA 11.1.1.7.12 (?) is available

$
0
0
OBIEE 11.1.1.7 new general bundle patch (number 11?), 11.1.1.7.151020 is available in Patch 21814325, it can be installed on any previous 11.1.1.7 release.


The Patch is platform specific and is available for:
  • Linux x86
  • Linux x86-64
  • MS Windows (64-bit)
You can read the readme file here.

Patch
Abstract
16913445
Oracle Business Intelligence Installer (BIINST) -same as previous patchset
19822893
Oracle Business Intelligence Publisher (BIP) -same as previous patchset
19825503
Enterprise Performance Management Components Installed from BI Installer 11.1.1.7.0 (BIFNDNEPM) -same as previous patchset
21815845
Oracle Business Intelligence Server (BISERVER)
21920526
Oracle Business Intelligence Presentation Services (BIPS)
19823874
Oracle Real-Time Decisions (RTD) -same as previous patchset
16997936
Oracle Business Intelligence ADF Components (BIADFCOMPS) -same as previous patchset
21920543
Oracle Business Intelligence Platform Client Installers and MapViewer
21235729
Oracle Business Intelligence Third Party - same as previous patchset

As usual there is a bonus patch:Patch 16569379  (1111770)- Dynamic Monitoring Service patch.It is platform dependent. If you installed it with 11.1.1.7.1 or later, that's enough. It's the same.


Bugs fixes 

Oracle Business Intelligence Server (BISERVER)

 Fix For Bug  20747758



P.S.

OBIEE patch numbers do it again.
After 11.1.1.7.140715 vs.
         11.1.1.7.150714;

It is 11.1.1.7.151020 vs.
       11.1.1.7.150120.

OBIEE 12c (12.2.1) is available

$
0
0
Oracle Business Intelligence 12c is available for downloadhere

It is available for:
  • Windows X86 -64bit
  • Linux X86-64bit
  • Oracle Solaris on SPARC 64-bit
  • Oracle Solaris on x86 64-bit
  • IBM AIX 64-bit
  • HP UX Itanium
System Requirements and Supported Platforms for Oracle Fusion Middleware 12c (12.2.1) is here.

The documentation is here .


What is new in 12c?

 Quite a lot.
First of all,everything regarding data visualization is new. See: User's Guide for Oracle Data Visualization 


I gathered here new features chapters from various 12c Books:

New features in Oracle BI EE 12c (12.2.1) include:
Enhancements to Graphs
In this release graphs have been enhanced as follows:
  • Ability to sort graph views by using a context menu (right-click). A new sort dialog is displayed when you right-click in a graph view in the "Analysis editor: Results tab" or on a dashboard page. You can right-click legend, data marker, and group-level (X axis) items to display a menu of interactions that includes options such as Sort and Drill (see "Right-Click Interactions in Views" for additional information).
Enhancements to Views
This release includes enhancements to various view types that can assist a content designer in creating more effective analyses faster, including those in the following list:
  • A new view type named heat matrix. Heat matrices are two-dimensional representations of data in which values are characterized by a gradient of colors. A simple heat matrix view provides an immediate visual summary of information that is well suited for analyzing large amounts of data and identifying outliers. See "Editing Heat Matrix Views" for additional information.
  • Ability to sort graph, heat matrix, pivot table, table, treemap, and trellis views by using a context menu (right-click). A new sort dialog is displayed when you right-click in a data view in the "Analysis editor: Results tab" or on a dashboard page (see "Right-Click Interactions in Views" for additional information).
Enhancements to Analyses
This release provides, for analyses, the ability to:
Enhancement to Scorecards
In this release, you can style and customize KPI Watchlists and every watchlist that appears in a scorecard. You can set custom styles for each watchlist and rename and change the order and visibility of the watchlist columns. See "Customizing Watchlists."
Enhancements to Dashboards
In this release, you can set advanced page properties to specify incoming navigation parameters you want to set on the current dashboard page. This enables you to control how parameters associated with the navigation actions are applied. See "Setting Advanced Page Properties."




New metadata repository features in Oracle BI EE 12c Release (12.2.1) include:
Logical Level Sequence Numbers for Time Dimensions
The Sequence Numbers tab has been added to the Logical Level dialog. The new tab allows you to add absolute or relative sequence numbers to time dimensions. These mappings provides direct column references in the Time dimension table, which creates a query that is easier for Oracle BI Server to execute against the data source. See "Adding Sequence Numbers to a Time Dimension's Logical Level" for more information.
DISPLAY | SORTKEY Syntax Supported in the SQL ORDER BY Expression
The Oracle BI Server now accepts the DISPLAY and SORTKEY keywords in the SQL ORDER BY expression. You can use the DISPLAY keyword to override a logical column's assigned sort order column. For more information, see "ORDER BY Clause Syntax."
Oracle Database Fast Application Notification and Fast Connection Failover Supported by Oracle BI Server
The Oracle BI Server supports the Fast Application Notification (FAN) event and Fast Connection Failover (FCF) Oracle Database configuration. Fast Connection Failover enables quick failover when the data source's Oracle database is not available. See "Oracle Database Fast Application Notification and Fast Connection Failover" for more information.
Generate Fragmented Aggregates in Aggregate Persistence
The aggregate persistence functionality has been enhanced to generate fragmented aggregates from a manually written aggregate specification. You can generate fragmented aggregates by adding a Where clause to the Logical SQL query's Create statement. See "Writing the Create Aggregates Specification" for more information.
New Command Line Utilities
Several command line utilities have been added. See the following topics for more information:


New system administration features and changes in Oracle BI EE 12c (12.2.1) include:
Invoking WLST From a Single Location
In previous releases, you invoked WLST from different locations, depending on whether you were using the commands for Oracle WebLogic Server, system components, or Java components such as Oracle SOA Suite. In this release, you invoke WLST from:
(UNIX) ORACLE_HOME/oracle_common/common/bin/wlst.sh
(Windows) ORACLE_HOME\oracle_common\common\bin\wlst.cmd
Oracle Home Location Redefined and No Middleware Home
Redefining of the Oracle home and elimination of the Middleware home. See "New And Deprecated Terminology for 12c" in Oracle Fusion Middleware Concepts.
OPMN is No Longer Used in Fusion Middleware
OPMN is no longer used in Oracle Fusion Middleware. Instead, system components are managed by the WebLogic Management Framework, which includes WLST, Node Manager and pack and unpack. See "What Is the WebLogic Management Framework" in Oracle Fusion Middleware Concepts.
Oracle Web Cache Not Part of Fusion Middleware
Oracle Web Cache is no longer part of Oracle Fusion Middleware.
Moving From Test To Production is Carried Out in a Different Way
The test to production operation is still possible however, the process is different from what was available in Oracle Business Intelligence Release 1 (11.1.1) as it applies solely to metadata (content, data model and authorization). For information, see Chapter 22, "Moving Oracle Business Intelligence Between Environments".
New Commands For Process Control
New process control commands replace the old start stop commands. For information, see Section 1.5.3, "Process Control Commands".
Managing Metadata In Business Intelligence Archive Files
All Oracle Business Intelligence metadata, including repository, Presentation Services catalog, and user authentication is stored in BAR archive files. The BAR file is a mechanism for managing or moving a self contained set of Oracle BI metadata between environments. For information, see Chapter 9, "Managing Metadata and Working with Service Instances".
Single Enterprise Install
In this release the Oracle Universal installer offers a single install type for your Enterprise which provides an Administration server, and a Managed server. For information, see Section 1.3, "What Is the Oracle Business Intelligence System Logical Architecture?" and Oracle Fusion Middleware Installation Guide for Oracle Business Intelligence.
Changes to Scaling Out
In this release the scale out procedures for Oracle Business Intelligence have changed. For information, see Chapter 3, "Scaling Your Deployment".
Simplified Configuration
Configuration files are no longer duplicated. Separate configuration files still exist for example, for Oracle BI Presentation Services and BI Server, but they are not duplicated in the case of a cluster. For information, see Section 8.3, "Configuring Oracle Business Intelligence System Settings".
Managing System Component Instances Using Commands
OBIS (BI Server) system component instances are separately managed in BI 12.2.1 using service instance commands. For information, see Section 9.2, "Managing Service Instances".
Collecting Diagnostic Bundles
A new script enables you to collect the diagnostic bundles needed by Oracle Support or Development to help resolve issues. For information, see Section 6.2, "Collecting Diagnostic Bundles".
Synchronizing Mid-Tier Database Connection Details Command
A new command enables you to synchronize mid-tier database connection details when they have changed. For information, see "Synchronize Mid-Tier Database Connection Details Command".


New security features in Oracle BI EE 12c (12.2.1) include:
BISystemUser and BISystem Removed
To simplify administration and configuration in this release Oracle Business Intelligence no longer requires a real user called BISystemUser (or equivalent) for internal communication. The system user concept is now deemed "virtual" and is represented by the credential oracle.bi.system/system.user, for which the values are securely randomly generated by the Configuration Assistant. Oracle BI components continue to use this credential for internal communication, backed by Oracle BI Security. The application role BISystem is also no longer present in the Policy Store, and will be removed from any upgraded 11g environment.
User GUIDs Removed
In this release user GUIDs have been removed to make administration easier. GUIDs are replaced with user names. There is no longer any need to refresh GUIDs as part of lifecycle operations. Your administrator is now responsible for ensuring that users leaving the system are cleaned up from Oracle Business Intelligence.
Database Security Store
In this release the Security Store (Policy and Credential Stores) is configured in a relational database rather than in a file. The database is the same as used by RCU. This change makes scaling easier, and makes clusters more reliable.
Easier SSL Configuration
In this release configuring SSL end to end is now less complex and uses offline commands.
The key differences in SSL support in this release (from 11g) are as follows:
  • SSL uses the WebLogic trust store
    No additional BI-specific trust configuration is required.
  • Offline commands
    There is no need to use Fusion Middleware Control UI to configure processes.
  • Diagnostics for WebLogic certificate issues
  • Higher security - TLSv1.2 only
  • Configuration is central and not intermingled with user configuration.
  • Supports advanced options with no risk of settings being overwritten.
Migrating Catalog Groups to Application Roles
In this release a new process enables you to migrate Catalog groups to application roles.




This section describes new features for Oracle BI EE 12c (12.2.1). It contains the following topics:

New Service

Oracle Business Intelligence 12c (12.2.1) includes the following new service:





Publisher



New features and changes for Oracle BI Publisher 12c Release 1 (12.2.1) include:

Generate Explain Plan from SQL Data Set

For data sets generated by SQL queries issued against the Oracle Database, you can now generate an Explain Plan to provide valuable information about the efficiency of your query. For more information, see Section 9.8.1, "Generate Explain Plan."

Best Practices Information

Poorly constructed data models can result in out-of-memory exceptions. Use these best practices guidelines to help you tune your data models for more efficient memory usage. See Chapter 9, "Performance Best Practices."





In Release 12c (12.2.1) the look and feel of the user interface is refreshed. You will notice new icons and some changes to the presentation of features on the Home page.



Naturally we can expect, the previously described 11.1.1.9 new features are included in 12c.
(here, here and here)



There are new tutorials:

OBIEE - Installing OBIEE 12c on my Windows box

$
0
0
I wanted to install the new OBIEE 12.2.1 on my Windows 7 box. Since I want to test the process of migration of OBIEE 11 to 12, I installed it side by side with my existing 11.1.1.9 installation.

If you want to see Linux installation description, you can see Andrew's post here. My installation seems to be smoother then his, despite the fact that Windows 7 doesn't seem to be supported according to the certification matrix (12.2.1 one).

The installation is of 4 steps:
  1. JDK installation (if needed)
  2. Weblogic Server installation
  3. BI Installation 
  4. Configuration

I downloaded JDK 8, Weblogic server and 2 BI files from here. After unzipping them I had the following:
I installed the JDK. It's better not to accept the default of installing in "Program Files\Java" folder because of the space in the name. I installed it in c:\Java.

Next I opened a Command Prompt as Administrator. Moved to the above folder (where the files are located) and run the Weblogic installation:
c:\Java\jdk1.8.0_65\bin\java.exe -jar fmw_12.2.1.0.0_infrastructure.jar
Accepted the defaults and pointed it to a new MWHOME I created (D:\MWHOME12, in my case).

Next I run the BI installation in the same folder:
 setup_bi_platform-12.2.1.0.0_win64.exe
And selected the same MWHOME for BI installation (D:\MWHOME12, in my case). 
Here are the next screens:





Now I can run the configuration part.
I used the same Command Prompt Window, and run config.cmd from MWHOME\bi\bin

I left the default, including Essbase (note the RTD installation is gone).

passed the prerequests:


Didn't change the location, but selected the domain user and password:

Allowed the wizard to created the RCU schema (note the options are: Oracle DB, Oracle RAC DB and MS SQL server). there is no standalone RCU anymore.

and updated the DB connection details (note, no need to add "as SYSDBA" for Oracle sys user)  :

Next left the defaults:

The options here are new and interesting:


 Saved the response file (PLEASE do it at some point, it's not fun to look for the ports and URLs later).


Now the process is running.

While the configuration passed well, the BI Startup at the bottom stayed for few very long minutes on 0%. Then jumped to 100% and finished. Made me check the log few times, during the period.


Now is the last screen. If you didn't save the URLs before. Do it now:


The OBIEE didn't start on it's own.
It is working:


I don't know if this should be so or it's because I have a previous entry of "Oracle Business Intelligence" in the "All Programs" menu, but nothing was added there.

To stop and start the BI server I can run start.cmd and stop.cmd from MWHOME\user_projects\domains\bi\bitools\bin\ (D:\MWHOME12\user_projects\domains\bi\bitools\bin\start.cmd in my case).

After the installation my default ports are 9502 for BI and 9504 for EM.
For Administration Tool installation, the port 9516 is used in the ODBC settings, in my case.


One first discovery I had after the installation. Since OBIEE 12 is pure HTML, we can copy/paste data from analysis, in all browsers I tested:





One more similar post: http://ssssupport.blogspot.co.il/2015/10/installing-oracle-obiee-12c-in-windows.html. Leela Madhav preferred to set up RCU separately.
And another by  Srinivas Malyala: OBIEE12c installation on windows7 machine






To the best of my knowledge the RCU should be installed on UTF8 type DB. Deliver BI try to work around it in OBIEE 12 RCU Avoid Hardstop Pre Reqs for AL32UTF8 Database Char Set.


Migration:
 After the installation, the natural step is to upgrade your existing OBIEE 11 to 12. It should be done from 11.1.1.7 latest patches or 11.1.1.9 and described in the post  Migrating from Oracle BI 11g to 12c by Shahed Munir.
And at allthingsorcl.blogspot.com here by @KeitaFK.
The Oracle migration guide is here.

Note the following from Oracle Support: OBIEE 12c Pre-Upgrade Alert: Before Upgrading To 12c, Ensure your 11g Catalog Is Upgraded To the Correct Version (Doc ID 2068965.1)
 

OBIEE 12c - Copy - Paste data from Analysis

$
0
0
We had a short discussion about the option to Copy / Paste of data from analysis to anywhere else.
In OBIEE 12c this option exists. There is a difference in it's behavior.

When working with Tables/Pivots in Fixed headers with scrolling content we are limited by OBIEE.


 I could only mark cells from one column and never mark Axis values in Pivot tables.

But the better abilities exist when switching to Content paging.

Now we can mark the Pivot/Table from outside the frame an select multiple values from multiple columns:


This works even in Pivots (bigger picture):

 
 

OBIEE 12c - File locations and meanings

$
0
0
The file locations in OBIEE12c are very different from previous versions. I'll update this page when I find something I consider interesting.


First few naming conventions:

I installed OBIEE and Weblogic under d:\MWHOME12.

ORACLE_HOME would be the installation Directory. (In my case d:\MWHOME12\).

BI_DOMAIN, sometimes called DOMAIN_HOME is ORACLE_HOME/user_projects/domains/bi (in my case it is  D:\MWHOME12\user_projects\domains\bi).


 SDD (Singelton Data Directory) is DOMAIN_HOME/bidata (for one host installation). (In my case it is   D:\MWHOME12\user_projects\domains\bi\bidata).




The catalog is located by default at  SDD/service_instances/ssi/metadata/content/catalog
(In my case it is D:\MWHOME12\user_projects\domains\bi\bidata\service_instances\ssi\metadata\content\catalog).

The Repository is (sort of) in SDD/service_instances\ssi\metadata\datamodel\customizations (In my case it is D:\MWHOME12\user_projects\domains\bi\bidata\service_instances\ssi\metadata\datamodel\customizations).

The entire handling of Repository and Catalog is different in 12c and is handled via BAR files. More about it later.


Configuration files are located under BI_DOMAIN/config/fmwconfig/biconfig
(In my case D:\MWHOME12\user_projects\domains\bi\config\fmwconfig\biconfig)
Each in it's folder. For example:
  • instaceconfig.xml is in BI_DOMAIN/config/fmwconfig/biconfig/OBIPS  
  • NQSConfig.ini          in BI_DOMAIN/config/fmwconfig/biconfig/OBIS




LOGS: Are in BI_DOMAIN/servers, each in his folder. For example sawlog.log is under BI_DOMAIN/servers/obips1/logs. All the logs are located in the same place, including Adminserver logs.

Not only the logs are here. You will found tmp, cache and other folders as well.





Most executable scripts are located in BI_DOMAIN/bitools/bin (D:\MWHOME12\user_projects\domains\bi\bitools\bin, in my case).


The start and stop scripts are here and also the script that collects diagnostic for Oracle Support (diagnostic_dump.cmd  for Windows and diagnostic_dump.sh for Linux).

WebLogic Scripting tool (wlst.cmd / wlst.sh) is located in Oracle_Home/oracle_common/common/bin (D:\MWHOME12\oracle_common\common\bin in my case.)

מסלול Business Analytics & Big Data בשבוע אורקל 2015

$
0
0
שבוע אורקל יתקיים השנה בשבוע המתחיל ב15 לנובמבר. ביומיים של השבוע שלאחר מכן, (22,23 לנובמבר) יתקיימו גם ימי After Event Workshops.

http://www.oracleweek.com/


טובית ואני ננהל את מסלול הBusiness Analytics  & Big Data. ניתן לראות את ההרצאות כאן:

השקענו גם בסרטון חינוכי באורך דקה ו21 שניות, המתאר את המסלול ומפרט את ההרצאות. ניתן לראות אותו כאן.

אישית, אני מעביר את Introduction to Data Science Concepts and Toolsומתארח אצל דני במערכות BI בעידן ה- Big Data - טכנולוגיות, מתודולוגיות, כלים ומגמות.

Enabling R and the relevant Analytics functions on OBIEE 12c

$
0
0
One of the great new features of OBIEE 12c is the ability to have advanced analytic functions based on R. "R is a widely used environment for statistical computing and graphics and can be used with many different datasources including external files or databases."
This post will only deal with enabling this functionality on the server. But first what we get?

The default functions are Cluster, Outlier, Regr, Trendline and Evaluate Script:

For example Regression:

User can add additional R Based functions based on your own scripts through Evaluate Script . 


When trying to run Analytics function by default, without the setup, for example: REGR("Base Facts"."Revenue", ("Base Facts"."Discount Amount"), ("Products"."Product Type", "Products"."Brand"), 'fitted', '') I get an error. 
The critical part of it is: State: HY000. Code: 47092. [nQSError: 47092] The value of R_EXECUTABLE_PATH is set to /usr/bin/R which is invalid. (HY000).

To solve it, I have to setup R in the environment.
 
We can set up 2 types of R. "Regular" R And Oracle R Enterprise, (that is part of the Oracle DB). Both option are described in: Administration Guide, chapter 17.11


In this post I will do the basic "regular" R setup on Windows box. The Unix / Linux steps are in the guide as well as the Oracle R Enterprise.


The R package is in ORACLE_HOME/bi/bifoundation/advanced_analytics/r-installer.tar.gz (on my computer it is D:\MWHOME12\bi\bifoundation\advanced_analytics).

Oracle recommends to download and install unzip and wget utilities:
I unzipped the r-installer.tar.gz in the same folder on the disk, got r-installer.tar file and unzipped it as well. Now I have a RInstaller folder at the same location ORACLE_HOME/bi/bifoundation/advanced_analytics/. 

R uses internet connection to download software and packages. That is why before installing Oracle recommends to set up the definitions in proxy.txt.  For example, if you are working from within Oracle intranet you should setupproxy=http://www-proxy.us.oracle.com:80 in that file.What should you do when you are in "regular" / not secured network? This part took me some time. Eventually, I opened a command windows and executed ipconfig (ifconfig in linux). I used the default Gateway ip. Assuming my default Gateway 192.1.1.1,  in my proxy.txt I used proxy=192.1.1.1:80 .(Another, not recommended and probably not supported option, would be to delete the first 2 lines in r-package-install.R)

 

Now comes the actual installation.
I opened a command line in Oracle_Home/bi/bifoundation/advanced_analytics/RInstaller and run set path=%path%;C:\Program Files (x86)\GnuWin32\bin (to add wget to the path).
Now  RInstaller.bat install

Next to install the R specific packages I'd run RInstaller.bat installpackages
It installs required R packages (forecast, mvoutlier, randomForest, RJSONIO, and matrixcalc). Running this command also installs the OBIEEAdvancedAnalytics R package that is in the Oracle_Home/bi/bifoundation/advanced_analytics/RInstaller/OBIEERPackage.

If your failed with text such as bellow, that means you have a problem with your proxy.txt.   

You are using Oracle's distribution of R. Please contact Oracle Support for any problems you encounter with this distribution.

> proxy<-read.table("proxy.txt", header=FALSE, sep="=", stringsAsFactors=FALSE)[
,2]
> Sys.setenv(http_proxy=proxy)
> install.packages("forecast", repos="http://cran.us.r-project.org")
...

Warning: unable to access index for repository http://cran.us.r-project.org/bin/
windows/contrib/3.1
Warning message:
package 'forecast' is not available (for Oracle Distribution of R version 3.1.1)



My R installation is now in C:\Program Files\R\R-3.1.1

 
Lets find NQSConfig.ini in BI_DOMAIN/config/fmwconfig/biconfig/OBIS = ORacle_home/user_projects/domains/bi/config/fmwconfig/biconfig/OBIS. Find there ADVANCE_ANALYTICS_SCRIPT section and update R_EXECUTABLE_PATH to point to the R exactable path: C:/Program Files/R/R-3.1.1/bin/x64/R



Restarted OBIS.
Now it’s working.


For Linux guidelines see PEAK Indicators blog here.


OBIEE 12c and VA - Playing with "BI Ask" (using the Home search to create reports in Visual Analyzer / Data Visualization)

$
0
0
During last few years, at Oracle Open World, an option named "BI Ask" was mentioned few times. I wanted to check it out now with OBIEE 12c. It should be part of the new BI component named Data Visualization, or some times Visual Analyzer. It's URL is http://myserver:9502/va/. I will call it VA and solve the name formalities problem.

In OBIEE Administration screen I Configured BI Search crawl:


I indexed Data Model (and Catalog). In my case I selected Index and not "Index Metadata Only", this option might take some time in real production environments (see user's guide appendix D for more details).

 Checked, the Monitor Crawls window to see the Success status


And I opened VA, directly using http://myserver:9502/va/. 


Another option to get to that screen is by selecting "go to the new Home Screen" from the (old) Home screen:

If you select  New / Visual Analyzer Project from OBIEE 12c, you will get into a new project window of VA with slightly different URL http://myserver:9502/va/project.jsp#.


In this case I want the first 2 options.
I don't know if it's a bug or a feature, but by first time entering VA requires login again.

I will use this line to create a report just by typing names and values. From what I understand, the same on mobile can be done by talking to the phone/tablet.

The next screens will show what I typed and selected as well as the outcome:

Typed "Rev" and selected Revenue from Sample Sales Base Facts:

 Result:


 Typed 2009 and selected Year Value:


The result:

Typed "Mon" and selected "Billed Month":
Result:

Typed "Comp" and selected Offices.Company (note graph options that appeared on the right part of screen):
I don't want all 3 companies so:



Any time I can change Visualization Type:

Zoom in:



Here is a Tree Map

And This is a Tag Cloud

Any Time I can continue working with all the features of VA:




More about VA, some other time. 
 

OBIEE - Selecting Date range from a Prompt Without breaking aggregation awareness

$
0
0
In the past I wrote about "OBIEE Prompts - Last Year, Last Month and Custom Dates", it works fine and eventually passes date-based filters.

OBIEE developers often use aggregation awareness.

My customer wanted to combine the two methods, having prompt for selecting various date ranges (last month, last year, last 7 days, Yesterday...), but to use the aggregated data when possible.
In our example I will have two tables of fake data, with similar structure. One will have the daily data and the other aggregated by month. Both will share the same logical table and each will be used based on the content of the query.

In aggregation awareness case, each time I pass a date parameter, the lower, less efficient, level of date fact will be used, so I can't simply use my above mentioned post.     

This post is written using OBIEE 12c, but should work with all previous versions. I'd be happy to hear if you have a better, more elegant, solution.

Starting point:
I have six tables: 
  • F_DAY - Daily Fact (Sales of 10 of product x, each day between 1-Oct-2015 and 4-Nov-2015. That 310 in October and 40 in November).

  • F_Month - Monthly Fact, (wrong summary of F_Day. The wrong data lets me see easier what table is used, without checking logs). I use first day of month to describe month.

  • D_Prods - Product dimension with one row of data (surprisingly, it's x)
  • DDATES - Date Dimension that include a row for each day with various column such as DDATE (the date), MMONTH (number in format YYYYMM), Day_Order (Number in format YYYMMDD) and many others. The last 2 are important.
  • Month - subset of DDATES that has only the month, and above level data.
  • BI_Time_Filter - Filter table (actually a view) with various date ranges and start date, start month, end date, end month columns in each. A column named History, when it's value is 'Y', we should use the F_Month fact.



-------------------------------------------------------------
The following section simply covers basic aggregation awareness. The only non-trivial part is setting various columns from dates tables (such as Day_Order) on the relevant dates hierarchy level.

To make it work I create a dates logical table and Dimension.

In the table there are two data sources, both date tables.
the Month table does not have date specific columns, all the rest are common.
Note, both DDATE and Day_order column are defined as day level.
For each source, I assign the relevant level of data based on the hierarchy, in the content tab. For example the Month source:
   














Similar will be done with the Fact tables. 2 sources mapped to same columns and different levels of data for dates dimension of the Table F in the Business Model:

 Here is the F_day Content tab set to Dates level of hierarchy (the F_Month has Month level) :











As a  result, Analysis on month level return the "wrong" results from F_Month, as desired:
When introducing date level the system switches to F_Day table, even for such a "minor" changes as adding filter DDates >= 1-Jan-1900 and, in my case, returns different results:

-------------------------------------------------------------
End of aggregation awareness section. 

 
Now I want to create a prompt on the periods selection table (BI_Time_Filter) and based on that selection create 3 hidden prompts. Those prompts should be dynamic in setting the filter for the relevant aggregation level. It should be similar to the methods used in "OBIEE Prompts - Last Year, Last Month and Custom Dates" mentioned above.

So I will create 3 prompts:
  • P_1 that has 2 possible values. Those are column formulas from dates Dimension: "DDATES"."MMONTHFULL" (the number of the month) and "DDATES"."DAY_ORDER" (the number of the day).
  • P_From that will accept starting number of month or Starting number of date from the prompt.
  • P_Till that will accept ending number of month or ending number of date from the prompt.


Now I just need to use a single filter in all my reports (I can save it and reuse it):
@{P_1}{"DDATES"."MMONTHFULL"} between @{P_From}{201510} and @{P_Till}{201511}

Or without default values:
@{P_1} between @{P_From} and @{P_Till}

for example for values
P_1= "DDATES"."MMONTHFULL"
P_From =201510
P_Till= 201511

I get:



While for 
P_1="DDATES"."DAY_ORDER"
P_From =20151020
P_Till= 20151104

I get:


The last part would be creating the above mentioned prompts.
I need the periods selection table (BI_Time_Filter) available as a single, not joined table in the presentation level. Covered in "OBIEE - Model based on one table" post before, I'll fast forward it.

(I duplicate the BI_Time_Filter table in Business Model, and set a join between the duplicates on Business level only, no need for physical, it's the same table. Next move to the presentation only the dimension copy of the two).

Create a simple Period selection prompt that updates a P_Period variable.

Based on that value create the hidden prompt with the above mentioned 3 lines.

Here is the first line:

With default selection:
SELECT case when "BI_TIME_FILTER"."HISTORY"='Y' then '"DDATES"."MMONTHFULL"' else '"DDATES"."DAY_ORDER"' end saw_1 FROM "test_algo" WHERE "BI_TIME_FILTER"."DISPLAY_MSG" = '@{P_Period}{Previous Month}';

I hide it in the dashboard properties, under Dashboard Filters and Variables:
Here is the result (once with 'This Month', that works on Day level data and once with 'This Year', with Month Level data):





















If you want to, you can further complicate it with additional options such as Custom Dates (with extra Prompt for the Custom Values)...

OBIEE 12c Interesting Support Documents

$
0
0

OBIEE - Using condition ? value-if-true : value-if-false format with Variables

$
0
0
When Evaluating variables in OBIEE formula we usually use the case statement. 
Something like case when '@{PV}'='Yes' then 'YES' else 'NO' end 
 Or case when @{PV}=1 then 'YES' else 'NO' end

Sometimes for clarity we would rather have the entire if-then-else embedded in the Variable curly brackets . In that case we can use the javascript / C ... syntax of
 condition ? value-if-true : value-if-false 

To repeat the previous examples:
'@{(PV=='Yes')? 'YES':'NO'}'

'@{(PV==1)? 'YES':'NO'}'

In both cases NO is the default.

Of course we are not limited to equal condition. All the options (!=, >, <, <=... ) are valid.

If you remember my Selecting columns dynamically saga, the following would work as well:

@{(PV=='Year')? '"Time"."Per Name Year"':'"Time"."Per Name Qtr"'}


P.S.
By the way, this is one of many interesting things I found in the great OBIEE knowledge source of OBIEE Samples VM.

OBIEE 12c Advanced Analytics Functions part 1. Introduction & Trendline

$
0
0
In OBIEE 12c there is a set of new Advanced Analytics functions. Antony Heljula wrote a great post (as usual) at PEAKindicators blog called Testing the new R functions on the OBIEE 12c SampleApp where he shows the basic 4 (Trendline, Clusters, Outliers and Regression). 
Oracle describes them in Users Guide Appendix C (starting C-58) covering them all. I will post here my experience with them, as part of my learning process. Usually I start with the example from the online help and check additional (documented and not documented) options, sometimes add my observation.

 I plan to do a 7 part daily series:
  1. OBIEE 12c Advanced Analytics Functions part 1. Introduction & Trendline (this one)
  2. OBIEE 12c Advanced Analytic part 2: BIN and WIDTH_BUCKET
  3. OBIEE 12c Advanced Analytic part 3: Forecast
  4. OBIEE 12c Advanced Analytic part 4: Cluster
  5. OBIEE 12c Advanced Analytic part 5: Outlier
  6. OBIEE 12c Advanced Analytic part 6: Regression
  7. OBIEE 12c Advanced Analytic part 7: EVALUATE_SCRIPT

I assume you already configured OBIEE to work with R (described here). If you use ORE, the R script identifies it and uses ORE specific scripts. I believe it should support larger amount of data this way.

There are 2 groups of functions, those done with R and internal ones:

Advance Analytics Internal Logical SQL Functions

  • Bin
  • Width_Bucket
  • Trendline

Advance Analytics External Logical SQL Functions

  • Forecast
  • Cluster
  • Outlier
  • Regr
  • Evaluate Script
This is consistent with what I see in the MWHOME\bi\bifoundation\advanced_analytics\RInstaller\OBIEERPackage folder, in the scripts of OBIEEAdvancedAnalytics_12.2.1.0 (no trendline, bin ...):


One last remark in the introduction. Most of the R packages statistical by nature. As a result they have a "use Random Seed" option. When doing so the result might not be always consistent (for example, slightly different clusters). Even more challenging, sometimes you can select various statistical algorithms, that return different result. While unthinkable in most traditional financial reports, this behavior is expected in data exploration cases. If this is new to you, well, welcome to data science.
(http://www.smbc-comics.com/?id=3485)


Trendline

I will start with Trendline in this post. While Trendlines are popular and rather simple, they are not always considered as the best statistical model (see VividCortex blog here).
Interesting enough, the Trendline is described as "Internal", what I understand as "not done by R". 

I start with the basic example and complicate a bit.

I'm using the default Sample Sales Subject Area, with date and revenue columns:



The Trendiline function in this case is: TRENDLINE("Base Facts"."Revenue", ("Time"."Calendar Date"), 'LINEAR','VALUE')
   
The meaning is quite simple: Trendline of Revenue over dates (I'll explain  'LINEAR','VALUE'  in a moment).

The result is:
If we limit the data to one year, we will have a nicer slope.


 Now let's see the complete Syntax from Oracle docs:

Syntax

TRENDLINE( <numeric_expr>, ( [<series>] ) BY ( [<partitionBy>] ),
<model_type>, <result_type>, [number_of_degrees] )




Where:
numeric_expr indicates the data to trend. This is usually a measure column. Note that this is the Y-axis. In our case, it was "Base Facts"."Revenue".
series indicates the X-axis. This is a list of <valueExp> <orderByDirection>, where <valueExp> is a dimension column and <orderByDirection> is ASC or DES. The default is ASC. Note that this cannot be an arbitrary combination of numeric columns. (I Didn't find the correct syntax to use the ASC/DES values.- Boris)
partitionBy indicates the control break for the trendline.
model_type indicates the type of model to use. Currently you can only specify LINEAR.
result_type indicates the type of output. You can specify VALUE or MODEL. VALUE returns the regression Y values given X in the fit. MODEL returns the parameters in a JSON format string.
number_of_degrees is used in polynomial models, only. This parameter is optional.




I added "Product Type" to the analysis, but not the trendline definition:


While it seems to work fine, it should be clear that the trendline definition is the same for all Product Types. 
If I want a specific Trendline for each Product Type, I should alter the Trendline function as well (by adding the  BY ("Products"."Product Type") to the formula) to:
TRENDLINE("Base Facts"."Revenue", ("Time"."Calendar Date") BY ("Products"."Product Type"), 'LINEAR','VALUE')


We can see the difference:




Changing the definition from Value to Model (TRENDLINE("Base Facts"."Revenue", ("Time"."Calendar Date"), 'LINEAR','MODEL')) returns:





We can the difference. It is the same Model for all Product Types ({"analyticmodel": {"identifier": "001", "name": "trendline", "type": "linear", "formula": "0.611644210520042*x+2230.67258977209"}}) for "Trendline without by" column, as expected.


As you can see, I use few Trendline columns in the same analysis.

But they must have the same X-axis. Or you will get:

[nQSError: 42072] There are multiple TRENDLINE functions in the query with different X-axis. 


This will happen if I try to have 2 Trendline functions such as:

TRENDLINE("Base Facts"."Revenue", ("Time"."Calendar Date"), 'LINEAR','Value')
 and
TRENDLINE("Base Facts"."Revenue", ("Time"."Per Name Year","Time"."Per Name Month") ,  'LINEAR','Value')


While in most of the cases the series in the function is date oriented, it can be just a Numeric Column (for example TRENDLINE("Base Facts"."Revenue", ("Products"."Product Number") ,  'LINEAR','Value'))


But it can't be text column. then we get an error:
[nQSError: 42058] The non-time dimension column in TRENDLINE X-axis is not numeric.

The Year and Month series were accepted in the following example (despite being text columns) because they are part of the date hierarchy:
TRENDLINE("Base Facts"."Revenue", ("Time"."Per Name Year","Time"."Per Name Month") ,  'LINEAR','Value')



A bunch of little, clean, trendline graphs (or trellis) might always seem interesting in an exacutive dashboard:




P.S.
Despite, what is said in documentation the ModelType value 'EXPONENTIAL' (instead of 'LINEAR') seems to return values, but only if you have something in the PartirionBy.


{"analyticmodel": {"identifier": "001", "name": "trendline", "type": "exponential", "formula": "1525.179067806*1.00056329831379^x"}}

 TRENDLINE("Base Facts"."Revenue", ("Time"."Calendar Date")  By ("Products"."Product Type"),  'exponential','Value')


OBIEE 12c Advanced Analytic part 2: BIN and WIDTH_BUCKET

$
0
0
this is the second part:

  1. OBIEE 12c Advanced Analytics Functions part 1. Introduction & Trendline
  2. OBIEE 12c Advanced Analytic part 2: BIN and WIDTH_BUCKET (this one)
  3. OBIEE 12c Advanced Analytic part 3: Forecast
  4. OBIEE 12c Advanced Analytic part 4: Cluster
  5. OBIEE 12c Advanced Analytic part 5: Outlier
  6. OBIEE 12c Advanced Analytic part 6: Regression
  7. OBIEE 12c Advanced Analytic part 7: EVALUATE_SCRIPT
  
In this post I will talk about 2 functions with similar but somewhat different functionality:  BIN and WIDTH_BUCKET.
Both functions aim is to define the bin / bucket the specific data entry belongs to. User can specify:
  • Using what column the binning should be done- binned expression (it's a numeric expression / usually a measure).
  • By what attributes it should be arranged; Please note the BY does not have the same meaning in both function.
  • Number of Bins / Buckets and the type of data returned (one of 3: the bin / bucket number, it's min or max point). 
  • In Bin function there is also a where condition option for the BIN.
The major difference is the different meaning of the BY parameter and the fact that the BIN function result is treated as a Dimension Attribute while WIDTH_BUCKET is not. 

The WIDTH_BUCKET itself is one more "secret function", not available in the function menu.
For some reason, the syntax of the 2 functions is different.

More on the difference is available in the documentation :

Note the following differences between the BIN and WIDTH_BUCKET functions:
  • Unlike the BIN function, the WIDTH_BUCKET function is not treated as a new dimensional attribute for the purposes of aggregation. Instead, the WIDTH_BUCKET function is applied on top of the query result similar to the other display functions such as RANK, TOPN, BOTTOMN, NTILE, PERCENTILE, MAVG, and MEDIAN.
  • The BY clause of the BIN function defines the grain at which the binned expression is evaluated prior to binning. If the binned expression is a measure, then the measure is grouped at the grain specified in the BY clause before being binned.
  • The BY clause of the WIDTH_BUCKET function defines the groups in the query result set over which the WIDTH_BUCKET calculation is applied. The buckets within different groups are calculated independently.
  • The BY clause of the BIN function is mandatory if the binned expression is a measure. Otherwise, for non-measure expressions, the BY clause is optional.
  • The BY clause is always optional in the WIDTH_BUCKET function. If the BY clause is omitted from the WIDTH_BUCKET function, then the function operates over the entire result set.
  • Use the BIN function when you want to compute a set of discrete buckets on top of a continuous valued attribute or measure and you want to treat that new set of discrete buckets as if it were a new dimension attribute that is intended to be included in the GROUP BY clause of other base measures in the query.
  • Use the WIDTH_BUCKET function when you want to compute a discrete set of buckets on top of an already aggregated query result set.
Simple examples of both functions, then the syntax and few more examples.

Simple Example


I'll use a "sample sales" based analysis With "Product Type", "Month" and "Revenue" columns.

To place the Revenue values into 4 bins based on all the attributes of the analysis, I can get the same results with the following BIN and WIDTH_BUCKET function:


BIN:
BIN("Base Facts"."Revenue"  BY "Products"."Product Type","Time"."Per Name Month" into 4 bins)

WIDTH_BUCKET:
WIDTH_BUCKET("Base Facts"."Revenue", NUMBER, 4) 



I had to define number of bins (4 in my case) in both functions (or get syntax error).

Few first differences:
The BIN function syntax is consistent with most Advanced Analytics in OBIEE while WIDTH_BUCKET is comma-based.
In BIN function I had to define the BY attributes (since "Revenue" is a measure) while I could omit them in WIDTH_BUCKET and accept the default.

Attempt to omit them in Bin function would cause an error in the Results Tab: 



Error Codes: OPR4ONWY:U9IM8TAC:U9IM8TAC:U9IM8TAC:U9IM8TAC:OI2DL65P
State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred. (HY000)
State: HY000. Code: 43113. [nQSError: 43113] Message returned from OBIS. (HY000)
State: HY000. Code: 43119. [nQSError: 43119] Query Failed: (HY000)
State: HY000. Code: 23058. [nQSError: 23058] The BIN expression requires a nonconstant BY clause if the expression being binned is a measure. (HY000)
SQL Issued: SELECT 0 s_0, "Sample Sales Lite"."Products"."Product Type" s_1, "Sample Sales Lite"."Time"."Per Name Month" s_2, BIN("Sample Sales Lite"."Base Facts"."Revenue" into 4 bins) s_3, DESCRIPTOR_IDOF("Sample Sales Lite"."Products"."Product Type") s_4, "Sample Sales Lite"."Base Facts"."Revenue" s_5, WIDTH_BUCKET("Sample Sales Lite"."Base Facts"."Revenue",NUMBER,4) s_6 FROM "Sample Sales Lite" FETCH FIRST 65001 ROWS ONLY

The difference in meaning of BY


Now I will aggregate both functions BY "Month" and will show the different way each function interpretation of the meaning of it:

BIN:
BIN("Base Facts"."Revenue"  BY "Time"."Per Name Month" into 4 bins)

WIDTH_BUCKET:
WIDTH_BUCKET("Base Facts"."Revenue", NUMBER, 4 by "Time"."Per Name Month")
The result is not the same at all:
 

WHY?

The meaning of BY "Month" in WIDTH_BUCKET is: Take individual rows of data in each month and arrange them in 4 buckets.

The meaning of BY "Month" in BIN is: Take the sum("Revenue" by "Month") and arrange the sum of month in 4 bins. So rows of the same month will have the same BIN "Revenue" by "Month" results.

To be sure I understand it I asked myself, what would be the result of
WIDTH_BUCKET("Base Facts"."Revenue", NUMBER, 4 by "Products"."Product Type", "Time"."Per Name Month")
Since all the data I have is combinations of the 2 attributes in the BY parameter, the function will return 1 for all the rows.
 
 

Returning


Before the formal syntax, what does the NUMBER parameter stands for in both functions?
It declares what the function should return. The Bin/Bucket number or the lower or the high value of the Bin/Bucket interval. The Possible values are: NUMBER, RANGE_LOW, RANGE_HIGH. we can have only one in each function. Here is the fist example with 3 WIDTH_BUCKET functions, each with a different Returning parameter:
 

And, naturally, the same result with BIN Function:
(For example, the BIN-High_Range formula is: BIN("Base Facts"."Revenue"  BY "Products"."Product Type","Time"."Per Name Month" into 4 bins Returning RANGE_HIGH) )
 



 The Syntax of BIN:


BIN(numeric_expr [BY grain_expr1, ..., grain_exprN] [WHERE condition]
INTO number_of_bins BINS [BETWEEN min_value AND max_value]
[RETURNING { NUMBER | RANGE_LOW | RANGE_HIGH }])

Where:
numeric_expr indicates the measure or numeric attribute to bin.
BYgrain_expr1, ..., grain_exprN indicates a list of expressions that define the grain at which the numeric_expr is calculated before the numeric values are assigned to bins. This clause is required for measure expressions and is optional for attribute expressions.
WHEREcondition indicates a filter condition to apply to the numeric_expr before the numeric values are assigned to bins.
INTOnumber_of_bins indicates the number of bins to return.
BETWEENmin_valueANDmax_value indicates the minimum and maximum values used for the end points of the outermost bins.
RETURNING indicates a filter condition to apply to the numeric_expr before the numeric values are assigned to bins. Note the following options:
  • RETURNING NUMBER indicates the return value should be the bin number (for example, 1, 2, 3, 4). This is the default condition.
  • RETURNING RANGE_LOW indicates the lower value of the bin interval.
  • RETURNING RANGE_HIGH indicates the higher value of the bin interval.

The Syntax of WIDTH_BUCKET:


The syntax of the WIDTH_BUCKET function is simpler than the BIN function, using simple comma-separated arguments instead of multiple clauses. The only nested clause supported by the WIDTH_BUCKET function is the BY clause, which is supported by all display functions including WIDTH_BUCKET, RANK, TOPN, and PERCENTILE.

WIDTH_BUCKET(numeric_expr, { NUMBER | RANGE_LOW | RANGE_HIGH },
number_of_bins, [min_value, max_value] [BY expr1, ..., exprN])

Where:
numeric_expr indicates the measure or numeric attribute to bin.
NUMBER indicated that the return value should be the bin number (for example, 1, 2, 3, 4). This is the default condition.
RANGE_LOW indicates the lower value of the bin interval.
RANGE_HIGH indicates the higher value of the bin interval.
number_of_bins indicates the number of bins to return. The default is 10.
min_value, max_value indicates the minimum and maximum values used for the end points of the outermost bins. If the min_value and max_value conditions are omitted, then the function determines the end points automatically
BYexpr1, ..., exprN indicates an optional list of expressions that define the groups in the query result set over which the WIDTH_BUCKET calculation is applied. The bucket intervals within different groups are calculated independently.
 



From the syntax we can see both functions have parameters (min_value, max_value) to exclude outermost points, if so desired. 




One last BIN function examples:

The "where" example of BIN function:


I'll use the following function:

BIN("Base Facts"."Revenue"  BY "Products"."Product Type","Time"."Per Name Month"  where "Time"."Per Name Year"='2010' into 4 bins)
Without any extra filtering.

The result is:


You can't see it in the picture, but as a result the data of the entire analysis is filtered to the Year 2010, since BIN has NULL values for any data that is not 2010. Turning on the "Include rows with Null values" option in the table properties would also return data with Null values in Revenue column.








OBIEE 12c Advanced Analytic part 3: Forecast

$
0
0
This is the third part:

  1. OBIEE 12c Advanced Analytics Functions part 1. Introduction & Trendline
  2. OBIEE 12c Advanced Analytic part 2: BIN and WIDTH_BUCKET
  3. OBIEE 12c Advanced Analytic part 3: Forecast (this one)
  4. OBIEE 12c Advanced Analytic part 4: Cluster
  5. OBIEE 12c Advanced Analytic part 5: Outlier
  6. OBIEE 12c Advanced Analytic part 6: Regression
  7. OBIEE 12c Advanced Analytic part 7: EVALUATE_SCRIPT
 (for 3-7 you need Enabling R and the relevant Analytics functions on OBIEE 12c )


The Forecast function is hiding with the Time Series Calculations (Unlike the Analytics). It makes sense since it includes time-series component.

The formal definition is:The FORECAST function creates a time-series model of the specified measure over the series using either Exponential Smoothing or ARMIA. This function outputs a forecast for the set of periods as specified by numPeriods.

I will use an example similar to what comes with the Online documentation and then will check the formal syntax and examine it further.

 In SampleSales based analysis with Year, Qtr, Revenue columns I will add a Forecast column. 

This is it's simplified code FORECAST("Revenue", ("Year" timeYear, "Qtr" timeQuarter),'forecast','modelType=arima;numPeriods=%1;predictionInterval=70;', 3)
This is the real function: FORECAST("Base Facts"."Revenue", ("Time"."Per Name Year" timeYear, "Time"."Per Name Qtr" timeQuarter),'forecast','modelType=arima;numPeriods=%1;predictionInterval=70;', 3)

The result is very interesting, we can see both seasonality and trend in the forecast:

I wanted a combined line of forecast and Revenue so I used something like: IFNULL("Revenue", "FORECAST")

This way the line looks nicer:


Now it's time to see the syntax and understand what we have (I made some modifications to the formal documentation).


Syntax

FORECAST(numeric_expr, ([series]), output_column_name, options
[runtime_binded_options])])

Where:
numeric_expr indicates the measure to forecast. For example, revenue data to forecast.

series indicates the time grain at which the forecast model is built. This is a list of one or more time dimension columns. If you omit series, then the time grain is determined from the query. You can omit the series only in cases you have only one Date/Time column in analysis. Even then you have the limitation described in appendix bellow. The series must fit the date columns in the Analysis. There is a predefined set of  column aliases, each should be named as seen in the example. (In my analysis they are 2 date columns, as a result, I can't treat them as optional otherwise I will get [nQSError: 42066] The parameter of TimeDimSeqNum function is not a time level key).
Example of Forecast with omitted  series in analysis with Qtr column only: FORECAST("Base Facts"."Revenue", (),'forecast','modelType=arima;numPeriods=%1;predictionInterval=70;', 3)
In all other cases, not using aliases might result in the problem described in the Appendix bellow.Here is a list of alias values and a table of their valid formats:
  • timeYear    
  • timeQuarter  
  • timeMonth   
  • timeWeek
  • timeDay
timeYear
timeQuarter
timeMonth
timeWeek
timeDay
2010
2010 Q1
2011 / 02
2011 / 33
"27/12/2012"
10
Q1 2010
2011/02
2011/33
"12/27/2012"
‘2010’
2010/Q1
02/ 2011
33 / 2011
"2012/12/27"

Q1/2010
02/2011
33/2011
"2012/27/12"

2010 / Q1
2011 - 02
2011 – 33
"27-12-2012"

Q1 / 2010
2011-02
2011-33
"12-27-2012"

2010-Q1
02 – 2011
33 - 2011
"2012-12-27"

Q1-2010
02-2011
33-2011
"2012-27-12"

2010 - Q1
2011 – Dec
2011 Week 04
"27.12.2012"

Q1 – 2010
2011-Dec
04 Week 2011
"12.27.2012"

Q1
Dec - 2011
04 Week, 2011
"2012.12.27"

1Q
Dec-2011
2011, Week 04
"2012.27.12"

1
2011 Dec
2011 Wk 04
"21-JAN-09" ORACLE DEFAULT DATE FORMAT DD-MON-RR

'2010 Q1'
Dec 2011
2011,Wk 04



Dec
2011, Wk 04



01
04 Wk 2011




04 Wk, 2011




04 Wk,2011




Week 04




04 Week




04 Wk




Wk 04




01

 


output_column_name indicates the output column. The valid values are 'forecast', 'low', 'high', and 'predictionInterval'. Why do the "low" and "high" exist? Forecasting is a delicate business. So knowing upper and lower limits of the prediction at the given confidence level might be important.

 
options indicates a string list of name/value pairs separated by a semi-colon (;). The value can include %1 ... %N, which can be specified in runtime_binded_options.
runtime_binded_options indicates a comma separated list of runtime-binded columns and options.



Now let's review our Function with the new knowldge:
FORECAST("Revenue", ("Year" timeYear, "Qtr" timeQuarter),'forecast','modelType=arima;numPeriods=%1;predictionInterval=70;', 3)

numeric_expr is "Revenue"
series is ("Year" timeYear, "Qtr" timeQuarter) note that adding a month column to the analysis and the series means changing to the following ("Year" timeYear, "Qtr" timeQuarter, "Month" timeMonth) each date column must be referenced with the proper name. Having non-consistency between the date columns in the analysis and in the series would result in the error: State: HY000. Code: 59144. [nQSError: 59144] Grain difference found....
There is no problem adding any other (not time) dimension to the analysis. For example


output_column_name in our case is 'forecast'

options is the string 'modelType=arima;numPeriods=%1;predictionInterval=70;', 3 where 3, at the end, is the runtime_binded_option, in this case indicating we want only 3 periods in the forecast. This is equivalent to 'modelType=arima;numPeriods=3;predictionInterval=70;'

modelType can be arima or ets. (ARIMA stands for 'Autoregressive integrated moving average', while ETS means 'Error,Trend,Seasonal'. In case you think Oracle gives you too many options, have a look here.)
Naturally, using different prediction algorithm return slightly different results:

Bellow is the same forecast function, once with Model Type Arima and once with ETS:



numPeriods is the number (Integer) of periods of forecast to predict.

Removing the numPeriods value (please note the ';' sign should remain) will return values for all possible future values in the database:
FORECAST("Base Facts"."Revenue", ("Time"."Per Name Year" timeYear, "Time"."Per Name Qtr" timeQuarter),'forecast','modelType=arima;;predictionInterval=70;')


predictionInterval is the confidence for the prediction. Values are 0 to 100 (not 100 itself, it returns nQSError: 43119 Confidence limit out of range), where higher values specify higher confidence.

Few more options:



Option Name
Description
Values
useBoxCox
If TRUE, then use Box-Cox transformation.
TRUE
FALSE
lambdaValue
The Box-Cox transformation parameter.
Ignore if NULL or when useBoxCox is FALSE.
Otherwise the data is transformed before the model is estimated.

modelParamIC
The information criterion (IC) to be used in the model selection.
"ic_auto"
"ic_aicc"
"ic_bic"
"ic_auto" (this is the default)


Some extra options are relevant only for the ETS model:

Option Name
Description
Values
trendDamp
If TRUE, then use damped trend. If NULL, then try both damped and non-damped trend and choose the one that is optimal.

errorType

additive("A")
multiplicative("M")
automatically selected ("Z")
trendType

none("N")
additive("A")
multiplicative("M")
automatically selected("Z")
seasonType

none("N")
additive("A")
multiplicative("M")
automatically selected("Z")


This post is getting too long.
To finish it, we might gain additional insight by looking at the internal documentation of the file obiee.TimeSeriesForecast.R, that comes with the installation (slightly shorter version), to discover some secrets like frequency.

# Function : obiee.Forecast.freeTimeFormat
#            STL Forecasting Model which applies ETS or Arima
#           Forecasts obtained by STL decomposition and then apply different models (ETS, Arima, ..) to seasonality adjusted data
#
# Arguments :
#
# dat                              dataframe          dataframe with date and target
#                                                     timeYear     : Can be YYYY or YY
#                                                     timeQuarter  : Can be YYYY Q%, Q% YYYY, YYYY / Q%, YY Q%, Q%
#                                                     timeMonth    : Can be
#                                                     timeWeek
#                                                     timeDay
#                                                     target
# numPeriods                numeric            number of periods to forecast
# predictionLevel          numeric            confidence level in percentage for prediction, default is 95.
# modelType                 string             "ets","arima"
# frequency                   string             "freq_hourly", "freq_daily",    "freq_weekdays", "freq_weekly",
#                                                     "freq_monthly", "freq_quarterly", "freq_yearly"
#                                                      Note: the frequency argument could be useful to derive the time date format,
#                                                          but it is not used in creating the time series object.
#                                                          for time series, we use obiee.Forecast.findFrequency() function to figure out the
#                                                          time series frequency.
# useBoxCox                        logical        if TRUE, use Box-Cox transformation. Otherwise, Box-Cox transformation is not used.default FALSE
# lambdaValue       numeric or NULL    Box-Cox transformation parameter. Ignored if NULL or when useBoxCox is FALSE.  Otherwise, data
#                                                     transformed before model is estimated.default NULL
#                                                     Note: we are not sure what to do when useBoxCox is true and lambdaValue = NULL
#                                                          currently it is
# trendDamp         logical          a parameter for ETS model. if TRUE, use damped trend.  if NULL, try both damped and non-damped trend
#                                                     and choose optimal. default NULL
# errorType                                 "A"/"M"/"Z"        a parameter for ETS model. additive("A"), multiplicative("M"),
#                                                     or automatically selected("Z"). default "Z"
# trendType                                "N"/"A"/"M"/"Z"    a parameter for ETS model. none("N"), additive("A"), multiplicative("M"),
#                                                     or automatically selected("Z"). default "Z"
# seasonType                             "N"/"A"/"M"/"Z"    a parameter for ETS model. none("N"), additive("A"), multiplicative("M"),
#                                                      or automatically selected("Z"). default "Z"
# modelParamIC    string             Information criterion (ic) to be used in model selection.
#                                                     Values are: "ic_auto", "ic_aicc", "ic_bic", default "ic_auto"
#
#
# Output Data Frame :
#
# Day or Month or other time unit:      Sequence of future time periods to forecast; the column name depends on the date format
# forecast                                            Point forecasts as a time series
# low                                                  Lower limits for prediction intervals at a given confidence level
# high                                                Upper limits for prediction intervals at a given confidence level
# confidenceLevel                             Confidence Level
#
# Assumptions:
# 1. assume only one column in the dataframe is target and it is non factor
# 2. assume the date column(s) are factors in dataframe
#######################################################################################################



Appendix


In the Sample Virtual Box the forecast Function sample page has 2 prompt (presentation variables):
numperiods - the number of periods to  predict
genTimeSeq -Generate future months even if they don't exist in warehouse (Yes / No)
The function is: forecast("Base Facts"."1- Revenue", ("Time"."T02 Per Name Month" @{(genTimeSeq=='Yes')? 'timeMonth':''}), 'forecast',  'numPeriods=@{numperiods}{5}' )

the interesting part here is the condition on the Presentation Variable: @{(genTimeSeq=='Yes')? 'timeMonth':''} If it equals 'Yes', return 'timeMonth' and then the forecast will automatically generate the missing month (2013/05 & 2013/06 in the example bellow). If not return '' and the both forecast month missing in the database will be aggregated to null line (lower picture).

Lets remove the Variable for clarity.

In the date dimension they are only 2013/01 - 2013/04.

This is  forecast("Base Facts"."1- Revenue", ("Time"."T02 Per Name Month" timeMonth), 'forecast',  'numPeriods=6 ) 


This is the same without timeMonth Alias:  forecast("Base Facts"."1- Revenue", ("Time"."T02 Per Name Month"), 'forecast',  'numPeriods=6) 




Viewing all 167 articles
Browse latest View live