Saturday, September 21, 2013

How to Write a Custom User Store Manager - WSO2 Identity Server 4.5.0

With this post I will be demonstrating writing a simple custom user store manager for WSO2 Carbon and specifically in WSO2 Identity Server 4.5.0 which is released recently. The Content is as follows,
  1. Use case
  2. Writing the custom User Store Manager
  3. Configuration in Identity Server
You can download the sample here.

Use Case

By default WSO2 Carbon has four implementations of User Store Managers as follows.
  • org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager
  • org.wso2.carbon.user.core.ldap.ReadOnlyLDAPUserStoreManager
  • org.wso2.carbon.user.core.ldap.ReadWriteLDAPUserStoreManager
  • org.wso2.carbon.user.core.ldap.ActiveDirectoryLDAPUserStoreManager
Let's look at a scenario where a company has a simple user store where they have kept customer_id, customer_name and the password (For the moment let's not worry about salting etc. as purpose is to demonstrate getting a custom user store into action). So the company may want to keep this as it is, as there may be other services depending on this and still wanting to have identities managed. Obviously it's not a good practice to duplicate these sensitive data to another database to be used by the Identity Server as then the cost of securing both databases is high and can guide to conflicts. That is where a custom User Store Manager comes handy, with the high extendibility of Carbon platform.

So this is the scenario I am to demonstrate with only basic authentication.

We have the following user store which is currently in use at the company.
CREATE TABLE CUSTOMER_DATA (
             CUSTOMER_ID INTEGER NOT NULL AUTO_INCREMENT,
             CUSTOMER_NAME VARCHAR(255) NOT NULL,
             PASSWORD VARCHAR(255) NOT NULL,
             PRIMARY KEY (CUSTOMER_ID),
             UNIQUE(CUSTOMER_NAME)
);


INSERT INTO CUSTOMER_DATA (CUSTOMER_NAME, PASSWORD) VALUES("pushpalanka" ,"pushpalanka");
INSERT INTO CUSTOMER_DATA (CUSTOMER_NAME, PASSWORD) VALUES("lanka" ,"lanka"); 

I have only two entries in user store. :) Now what we want is to let these already available users to be visible to Identity Server, nothing less, nothing more. So it's only the basic authentication that User Store Manager should support, according to this scenario.

Writing the custom User Store Manager

There are just 3 things to adhere when we writing the User Store Manager and the rest will be done for us.

  • Implement the 'org.wso2.carbon.user.api.UserStoreManager' interface
There are several other options to do this, by implementing 'org.wso2.carbon.user.core.UserStoreManager' interface or extending 'org.wso2.carbon.user.core.common.AbstractUserStoreManager' class, as appropriate. In this case as we are dealing with a JDBC User Store, best option is to extend the existing JDBCUserStoreManager class and override the methods as required.
CustomUserStoreManager extends JDBCUserStoreManager 

@Override
    public boolean doAuthenticate(String userName, Object credential) throws UserStoreException {

        if (CarbonConstants.REGISTRY_ANONNYMOUS_USERNAME.equals(userName)) {
            log.error("Anonymous user trying to login");
            return false;
        }

        Connection dbConnection = null;
        ResultSet rs = null;
        PreparedStatement prepStmt = null;
        String sqlstmt = null;
        String password = (String) credential;
        boolean isAuthed = false;

        try {
            dbConnection = getDBConnection();
            dbConnection.setAutoCommit(false);
            sqlstmt = realmConfig.getUserStoreProperty(JDBCRealmConstants.SELECT_USER);

            prepStmt = dbConnection.prepareStatement(sqlstmt);
            prepStmt.setString(1, userName);

            rs = prepStmt.executeQuery();

            if (rs.next()) {
                String storedPassword = rs.getString("PASSWORD");
                if ((storedPassword != null) && (storedPassword.trim().equals(password))) {
                    isAuthed = true;
                }
            }
        } catch (SQLException e) {
            throw new UserStoreException("Authentication Failure. Using sql :" + sqlstmt);
        } finally {
            DatabaseUtil.closeAllConnections(dbConnection, rs, prepStmt);
        }

        if (log.isDebugEnabled()) {
            log.debug("User " + userName + " login attempt. Login success :: " + isAuthed);
        }

        return isAuthed;

    }

  • Register Custom User Store Manager in OSGI framework
This is just simple step to make sure new custom user store manager is available through OSGI framework. With this step the configuration of new user store manager becomes so easy with the UI in later steps. We just need to place following class inside the project.

/**
 * @scr.component name="custom.user.store.manager.dscomponent" immediate=true
 * @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService"
 * cardinality="1..1" policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"
 */
public class CustomUserStoreMgtDSComponent {
    private static Log log = LogFactory.getLog(CustomUserStoreMgtDSComponent.class);
    private static RealmService realmService;

    protected void activate(ComponentContext ctxt) {

        CustomUserStoreManager customUserStoreManager = new CustomUserStoreManager();
        ctxt.getBundleContext().registerService(UserStoreManager.class.getName(), customUserStoreManager, null);
        log.info("CustomUserStoreManager bundle activated successfully..");
    }

    protected void deactivate(ComponentContext ctxt) {
        if (log.isDebugEnabled()) {
            log.debug("Custom User Store Manager is deactivated ");
        }
    }

    protected void setRealmService(RealmService rlmService) {
          realmService = rlmService;
    }

    protected void unsetRealmService(RealmService realmService) {
        realmService = null;
    }
}


  • Define the Properties Required for the User Store Manager
There needs to be this method 'getDefaultProperties()' as follows. The required properties are mentioned in the class 'CustomUserStoreConstants'. In the downloaded sample it can be clearly seen how this is used.
@Override
    public org.wso2.carbon.user.api.Properties getDefaultUserStoreProperties(){
        Properties properties = new Properties();
        properties.setMandatoryProperties(CustomUserStoreConstants.CUSTOM_UM_MANDATORY_PROPERTIES.toArray
                (new Property[CustomUserStoreConstants.CUSTOM_UM_MANDATORY_PROPERTIES.size()]));
        properties.setOptionalProperties(CustomUserStoreConstants.CUSTOM_UM_OPTIONAL_PROPERTIES.toArray
                (new Property[CustomUserStoreConstants.CUSTOM_UM_OPTIONAL_PROPERTIES.size()]));
        properties.setAdvancedProperties(CustomUserStoreConstants.CUSTOM_UM_ADVANCED_PROPERTIES.toArray
                (new Property[CustomUserStoreConstants.CUSTOM_UM_ADVANCED_PROPERTIES.size()]));
        return properties;
    } 

The advanced properties carries the required SQL statements for the user store, written according to the custom schema of our user store.
Now all set to go. You can build the project with your customization to the sample project or just use the jar in the target. Drop the jar inside CARBON_HOME/repository/components/dropins and drop mysql-connector-java-<>.jar inside CARBON_HOME/repository/components/lib. Start the server with ./wso2carbon.sh from CARBON_HOME/bin. In the start-up logs you will see following log printed.

INFO {org.wso2.sample.user.store.manager.internal.CustomUserStoreMgtDSComponent} -  CustomUserStoreManager bundle activated successfully.

Configuration in Identity Server

In the management console try to add a new user store as follows.
 In the shown space we will see our custom user store manager given as an options to use as the implementation class, as we registered this before in OSGI framework. Select it and fill the properties according to the user store.


Also in the property space we will now see the properties we defined in the constants class as below.
If our schema changes at any time we can edit it here in dynamic manner. Once finished we will have to wait a moment and after refreshing we will see the newly added user store domain, here I have named it 'wso2.com'. 
So let's verify whether the user are there. Go to 'Users and Roles' and in Users table we will now see the users details who were there in the custom user store as below.

 If we check the roles these users are assigned to Internal/everyone role. Modify the role permission to have 'login' allowed. Now if any of the above two users tried to login with correct credentials they are allowed.
So we have successfully configured Identity Server to use our Custom User Store without much hassel.

Cheers!

Ref: http://malalanayake.wordpress.com/2013/04/03/how-to-write-custom-jdbc-user-store-manager-with-wso2is-4-1-1-alpha/

Note: For the updated sample for Identity Server - 5.0.0, please use the link, https://svn.wso2.org/repos/wso2/people/pushpalanka/SampleCustomeUserStoreManager-5.0.0/

Google

Sunday, September 08, 2013

WSO2 Identity Server 4.5.0 - User Store Management

In this post we will be going through the high level view of user management in WSO2 Carbon Products from Kernel 4.2.0 on. Specifically in WSO2 IS 4.5.0 which is based on this Kernel. These versions are armed with the capability to configure user stores at run time. 

Org.wso2.carbon.user.core is the  OSGI component responsible for handling users in Carbon products. There we have the concept of 'User Realm' which is a collection of users with attributes. It consists of following four aspects,
  • Use store management
  • Authorization Management
  • Claim management
  • Profile configuration management
You can get a clear picture of these 4 aspects from this blog, http://xacmlinfo.org/2012/06/21/user-core-concepts-in-wso2-identity-server/ . Here we will see into the improvements done in User Store Management aspects with the newly released version. It provides the capability to configure user stores at run time, even in a clustered mode as described in this previous post by myself, using a convenient UI. Following diagram shows how it happening.

In the implementation, once we enter the user name and password, those are sent to User Store Manager(Taken from User Realm of the Tenant according to the user name) to authenticate. 
  • If user is as 'user1' user realm of super tenant is used. If user is as 'user1@wso2.com', User realm of tenant 'wso2.com' is used. In any case the flow is same that, first Primary user store manager checks for a matching user with same credentials.
  • If user name is correct, but password is wrong still it will not issue a decision on authentication, but continue to check. If there is a matching user it will return and go for the next step in the flow which is authorization. 
  • If a matching user is not found in primary user store, then the secondary user store manager is used and it will look in secondary user store whether a matching user exists. Like wise this will go till the end of user store manager chain. at this point the user is not authenticated and will provide the info that authentication is failed for provided credentials.
This procedure not new in the latest version.  Plugging user store managers at run time is what is new. For this we have a UI which can be used to create the configuration file. Once this user store management configuration file is dropped into relevant folder 'Deployment Manager' is triggered and it updates the chain accordingly.
For super tenant files goes to,
                    CARBON_HOME/repository/deployment/server/userstores/
For a general tenant files goes to,
                    CARBON_HOME/repository/tenants/<tenantID>/userstores

How Configurations are Populated in Cluster

The user store configurations are populated in a cluster using 'SVN based deployment synchronizer' component of WSO2 Carbon. Once this is correctly enabled in the cluster, the modifications we do in the primary node are committed to the SVN repo. Once the committing is done this node sends a cluster message so that other nodes can check it out from the svn repo. Then the modifications are checked out to the relevant folders. So with this modification, the 'Deployment Manager' of each node is triggered and flow in a single node will start.

You can try this out following this post.

Cheers!

Cluster mode - User Store Management Configuration at Run Time


We can even simply try this out with following simple steps in WSO2 Identity Server.

In the extracted pack go to, 
  • CARBON_HOME/repository/conf/axis2/axis2.xml and enable clustering
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent"
                enable="true"> 

  • CARBON_HOME/repository/conf/carbon.xml and set-up deployment synchronizer,

 <DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>true</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://svnrepo.example.com/repos/</SvnUrl>
        <SvnUser>username</SvnUser>
        <SvnPassword>password</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

This is our primary node in the cluster. Now take two copies of this extracted folder and change following in carbon.xml,

<Offset>1</Offset>
<DeploymentSynchronizer>
        <Enabled>true</Enabled>
        <AutoCommit>false</AutoCommit>
        <AutoCheckout>true</AutoCheckout>
        <RepositoryType>svn</RepositoryType>
        <SvnUrl>http://svnrepo.example.com/repos/</SvnUrl>
        <SvnUser>username</SvnUser>
        <SvnPassword>password</SvnPassword>
        <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>
I changed port offset as I will be running all the server instances in local machine. So port offset is set to 1 in one copy and it is set to 2 in the other copy. Other change is we are only letting the Primary to commit automatically to SVN repo, but not other nodes, hence auto-commit is set to false.

Now let's start all the 3 servers. Once started, follow this post being in Primary node. In a moment we will see the configurations are replicated to other two nodes as well.

Cheers!

Thursday, September 05, 2013

Deploying Identity Server over a JDBC Based User Store

With this post I am to demonstrate how to configure WSO2 Identity Server with a JDBC user store. For the demonstration I am using a MySQL user store, but same procedure applies to any other JDBC user store as well.
My environment is,
OS - Ubuntu 12.10
Java - 1.6
WSO2 IS 4.5.0
  1. Setting up MySQL database
  2. User Store Configuration in IS - Primary
  3. User Store Configuration in IS - Secondary
(I am referring to extracted wso2is folder as CARBON_HOME in this post)

Setting up MySQL database

We need MySQL running at first. This post will be helpful in setting up the MySQL database, if it's not already done. Once MySQL is running we have to set up the database as required by the Identity Server. The server packs the necessary sql scripts within itself, which can be located at CARBON_HOME/dbscripts. 

Let's login to MySQL server and execute the following,
Create a database,
mysql> create database JDBC_demo_user_store;
Check out the creation,
mysql> show databases; 
Then use the sql script and set up the database,
mysql> use JDBC_demo_user_store;
mysql> source <path_to>/wso2is-4.5.0/dbscripts/mysql.sql; 
This will run the queries in the SQL scripts and set up the required tables.
Now if we enter the commands following outputs will be shown.
mysql> show tables;

Now we are done with setting up the database. We can go ahead and ask Identiy Server to use it.

Note: Before going into the following steps we also need to add the mysql-jdbc connector to Identity Server. You can download it from here and drop it into CARBON_HOME/repository/components/lib.

User Store Configuration in IS - Primary 

Identity Server uses embedded H2 database to keep permission details etc. and the data source details of it resides in CARBON_HOME/repository/conf/datasources/master-datasources.xml. We can add data-source details of our new JDBC user store here as well. Here is the master-datasources.xml file according to my set-up.

<datasource>
            <name>JDBC_demo_user_store</name>
            <description>The datasource used for JDBC_demo_user_store</description>
            <jndiConfig>
                <name>jdbc/JDBC_demo_user_store</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://localhost:3306/JDBC_demo_user_store</url>
                    <username>root</username>
                    <password>pushpalanka2</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
</datasource>
The  Primary configuration for user store resides at CARBON_HOME/repository/conf/user-mgt.xml file. By default this is pointing to a embedded ReadLDAPUserStoreManager. Now we are to change it to be a JDBCUserStoreManager. So let's comment out the default one and uncomment JDBCUserStoreManager. Now we will have a user-mgt.xml file similar to this, with the <Property name="dataSource"></Property>  property being set to what is given at datasource. If we want, we can modify these properties as we want, according to the context.

Now the configurations are over. Let's start the server with bin/wso2server.sh. Once started if we go ahead and add user to the 'Primary' domain.


























Now if we go and check the UM_USER table created in our database, it will list user as well.


User Store Configuration in IS - Secondary

Now let's see how we can use that same MySQL user store as a secondary user store in IS. This is pretty easy that we can do the whole thing via UI, without any modification to the above default configurations in master-datasources.xml or user-mgt.xml. We have to add driver name, URL, user name and password here as mandatory properties which we previously gave at master-datasources.xml.



























Once added it will be shown in the available user stores list. It intuitive to define a user store manager in UI, but if you want more details, you can refer this post. If we want we can also edit the optional properties too. The advanced section carries the SQL statements required for JDBC user store manager.

Advanced Option: If we are editing database structure(sql script), we need to update these SQL queries according to that schema, using this Advanced option.

Now if we go and try add a new user, we will see this secondary domain as well.

We can see the users getting added in the database as same as it was in the Primary user store, if we select this domain and add the users.

Cheers!

Wednesday, September 04, 2013

Getting Started with MySQL

This is a simple beginners guide to use MySQL in linux, from installation to querying the databases.
  1. Installation
  2. Login
  3. Databases and tables

Installation

First let's make sure our package management tools are up-to date. For that run the following commands in command line.

sudo apt-get update
sudo apt-get dist-upgrade
Once it finishes update and upgrading, we can install MySQL with following command.
sudo apt-get install mysql-server mysql-client
This will take a moment to install and now we are ready to go. 

Login

At first start up MySQL server is not set up with a password for root and we can login with,
mysql -u root -p 
If we are setting the password for the first time we can use following to set-up,
mysqladmin -u root -p NEWPASSWORD   
If we want to change a previously set password following command can be used,
mysqladmin -u root -p'oldpassword' password newpassword

Databases and Tables

First we should login to MySQL server with,
mysql -uroot -p<password>
Then it will point us to mysql console as follows where we can run queries,
mysql>
To see the available databases,
show databases;

To see the available tables inside a database,
use <database_name>;
show tables;
To see the field formats of the table,
describe <table_name>; 
To delete database/tables,
drop database if exists <database_name>  (for a table use table name)
Just like that, we can also run SQL queries like "SELECT * FROM user;" which will print the result in console.

If we have an .sql script to set up the databases or tables, we can just run it with,

source <path_to_script_file>;


If we want to import a large database then following is recommanded,
mysql -u root -p database_name < database_dump.sql
Cheers!

Monday, September 02, 2013

Implemention of Support for Mutiple User Store Configuration at Run Time (A touch on the Beauty of WSO2 Carbon Architecture)

As I have shared in the previous post, WSO2 IS 4.5.0 version is released with added support for dynamic configuration of multiple user stores. While implementing this piece of component, I could touch some beautiful areas of WSO2 Carbon architecture,  which is known to be inherently dynamic and flexible. With this post I am to list those characteristics of Carbon platform that came handy in this implementation.The content of this post is,

  1. How dynamic User Store Configuration happens
  2. Carbon characteristics that facilitated rapid development

How dynamic User Store Configuration happens

Following figure highlights the flow of a new user store configuration.


  • The super admin or tenant admin can add user stores through the UI, to own domain. We have allowed dynamic configurations only for secondary user stores and 'Primary' user store is not configurable at run time. This was because it is available for all tenants and allowing changes to it's configuration at run time can guide to instability of system. With this limitation we have been able to keep the design simple and avoid some crucial run time complexities that may have occurred. (eg: Primary user store keeps super admin data used to sign in and if super admin himself changes the configuration of Primary user store, the status of the system in between, is not stable.) So the Primary is treated as a static property in the implementation that is a basic requirement to run the system properly.
  • These secondary user stores can be added in two ways, via the UI or directly dropping it in the corresponding location as shown in the figure. Anyway the recommended way is to use from the above is, using the UI, as it will mostly avoid us from putting wrong configuration files and guide us to do it correct. If we are providing it writing xml manually following factors needs to fulfilled. 
    1. The domain name should match the file name (wso2.com --> wso2_com.xml)
    2. All mandatory fields required by the User Store Manager implementation should be provided as properties. (UI it self has guidance for this or we can refer the documentation of WSO2 IS.)
  • If the configuration files are added though the UI, they will be saved in the locations given, according to the tenant. Now on, the deployer (an Axis2 custom deployers) will take care of it and update everything accordingly. The deployer is polling the 'userstores' folder to detect any changes and as soon as it is aware of an event, required update is called. The deployer will detect the changes with an upper limit of 15 seconds which means there will be a little delay to see the updates in the UI. Refreshing the page after this moment is wait will make the changes visible in the UI.
So what happens inside, after deployer detects the modification?

  • According to the done modification(add/delete/enable/disable/edit), the deployer will identify the events(Using the details of the modified file and whether it's deploy or undeploy). Then it lets the deployement manager to update the User Store Manager chain according the modification. If it's an addition the new user store manager will be added at the end of the chain. If it's a delete, the chain will be broken at the point and tail part will be added to head. Other modification will be effective in the chain as it is, without any effect to the order. 
  • So when a user comes and submit his/her credentials the authenticator goes through the chain and check for a matching user in own chain of tenant. If it's found in the user chain with matching credentials then the assigned roles are checked for authorization, which allows users to perform permitted actions. 

Carbon characteristics that facilitated rapid development 

  • Clear separation of Front end/Back end (SOA interfaces)
With this clear separation it was easy to re-use what has been already implemented that are also useful in this implementation. Also regarding this component, could see a clear separation of UI and back end that simplified the design. UI used the stub classes to talk to the backend component which talked to other back-end components and delivered the output to UI. So if we want to just consume the API at sometime, it is already available
  • Out the box support for multi-tenancy
After writing the component for Super tenant there was no more significant effort to make it work in a multi-tenanted environment. Once it runs fine for super tenant, it is running fine in multi-tenant environment too. So this can be available be run anywhere, cloud or on-premise without a single modification.
  • Clustering
How will this work in a cluster is our next question. So all these needed to be replicated in all the nodes. Then there is WSO2 Carbon feature that can be used to synchronize the nodes, 'Deployment Synchronizer' which has options to be based on SVN or registry. In the this implementation we used SVN based deployment synchronizer and target was achieved. Not a single line of code needed for this.
  • Extendibility
WSO2 Carbon is also called 'Eclipse for Servers' with it's high extendibility with the OSGI run time. This comes very useful regarding this component that after sometime if we wanted to add our own custom user store manager implementation like for Apache Cassandra it is also possible without any hassle. We just have to implement the provided UserStoreManager interface or extend the AbstractUserStoreManager class, write the customized code and pack it into a bundle (sample). Once this is dropped into CARBON_HOME/repository/components/dropins the run time will detect it at start-up and even show it in the UI for configurations.

Also when it was needed to detect the changes of configuration files, there was this custom deployer support inherited from Axis2 that made the life easier. We just had to extend the provided implementation pointing to the folder path to poll, write what we want do at new file addition(deploy)/file deletion(undeploy) and it was doing the job. 

Finally, it was so nice to see that all these components inter-operate with each other to satisfy the requirement in the exact expected way.

Cheers!