Apache Shiro authentication for Apache Zeppelin
Overview
Apache Shiro is a powerful and easy-to-use Java security framework that performs authentication, authorization, cryptography, and session management. In this documentation, we will explain step by step how Shiro works for Zeppelin notebook authentication.
When you connect to Apache Zeppelin, you will be asked to enter your credentials. Once you logged in, then you have access to all notes including other user's notes.
Important Note
By default, Zeppelin allows anonymous access. It is strongly recommended that you consider setting up Apache Shiro for authentication (as described in this document, see 2 Secure the Websocket channel), or only deploy and use Zeppelin in a secured and trusted environment.
Security Setup
You can setup Zeppelin notebook authentication in some simple steps.
1. Enable Shiro
By default in conf, you will find shiro.ini.template, this file is used as an example and it is strongly recommended
to create a shiro.ini file by doing the following command line
cp conf/shiro.ini.template conf/shiro.ini
For the further information about shiro.ini file format, please refer to Shiro Configuration.
2. Start Zeppelin
bin/zeppelin-daemon.sh start #(or restart)
Then you can browse Zeppelin at http://localhost:8080.
3. Login
Finally, you can login using one of the below username/password combinations.
[users]
admin = password1, admin
user1 = password2, role1, role2
user2 = password3, role3
user3 = password4, role2
You can set the roles for each users next to the password.
Groups and permissions (optional)
In case you want to leverage user groups and permissions, use one of the following configuration for LDAP or AD under [main] segment in shiro.ini.
activeDirectoryRealm = org.apache.zeppelin.realm.ActiveDirectoryGroupRealm
activeDirectoryRealm.systemUsername = userNameA
activeDirectoryRealm.systemPassword = passwordA
activeDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COM
activeDirectoryRealm.url = ldap://ldap.test.com:389
activeDirectoryRealm.groupRolesMap = "CN=aGroupName,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"group1"
activeDirectoryRealm.authorizationCachingEnabled = false
activeDirectoryRealm.principalSuffix = @corp.company.net
ldapRealm = org.apache.zeppelin.realm.LdapGroupRealm
# search base for ldap groups (only relevant for LdapGroupRealm):
ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COM
ldapRealm.contextFactory.url = ldap://ldap.test.com:389
ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COM
ldapRealm.contextFactory.authenticationMechanism = simple
also define roles/groups that you want to have in system, like below;
[roles]
admin = *
hr = *
finance = *
group1 = *
Configure Realm (optional)
Realms are responsible for authentication and authorization in Apache Zeppelin. By default, Apache Zeppelin uses IniRealm (users and groups are configurable in conf/shiro.ini file under [user] and [group] section). You can also leverage Shiro Realms like JndiLdapRealm, JdbcRealm or create our own.
To learn more about Apache Shiro Realm, please check this documentation.
We also provide community custom Realms.
Note: When using any of the below realms the default password-based (IniRealm) authentication needs to be disabled.
Active Directory
activeDirectoryRealm = org.apache.zeppelin.realm.ActiveDirectoryGroupRealm
activeDirectoryRealm.systemUsername = userNameA
activeDirectoryRealm.systemPassword = passwordA
activeDirectoryRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/conf/zeppelin.jceks
activeDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COM
activeDirectoryRealm.url = ldap://ldap.test.com:389
activeDirectoryRealm.groupRolesMap = "CN=aGroupName,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM":"group1"
activeDirectoryRealm.authorizationCachingEnabled = false
activeDirectoryRealm.principalSuffix = @corp.company.net
Also instead of specifying systemPassword in clear text in shiro.ini administrator can choose to specify the same in "hadoop credential".
Create a keystore file using the hadoop credential commandline, for this the hadoop commons should be in the classpath
hadoop credential create activeDirectoryRealm.systempassword -provider jceks://file/user/zeppelin/conf/zeppelin.jceks
Change the following values in the Shiro.ini file, and uncomment the line:
activeDirectoryRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/conf/zeppelin.jceks
LDAP
Two options exist for configuring an LDAP Realm. The simpler to use is the LdapGroupRealm. How ever it has limited flexibility with mapping of ldap groups to users and for authorization for user groups. A sample configuration file for this realm is given below.
ldapRealm = org.apache.zeppelin.realm.LdapGroupRealm
# search base for ldap groups (only relevant for LdapGroupRealm):
ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COM
ldapRealm.contextFactory.url = ldap://ldap.test.com:389
ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COM
ldapRealm.contextFactory.authenticationMechanism = simple
The other more flexible option is to use the LdapRealm. It allows for mapping of ldapgroups to roles and also allows for role/group based authentication into the zeppelin server. Sample configuration for this realm is given below.
[main]
ldapRealm=org.apache.zeppelin.realm.LdapRealm
ldapRealm.contextFactory.authenticationMechanism=simple
ldapRealm.contextFactory.url=ldap://localhost:33389
ldapRealm.userDnTemplate=uid={0},ou=people,dc=hadoop,dc=apache,dc=org
# Ability to set ldap paging Size if needed default is 100
ldapRealm.pagingSize = 200
ldapRealm.authorizationEnabled=true
ldapRealm.contextFactory.systemAuthenticationMechanism=simple
ldapRealm.searchBase=dc=hadoop,dc=apache,dc=org
ldapRealm.userSearchBase = dc=hadoop,dc=apache,dc=org
ldapRealm.groupSearchBase = ou=groups,dc=hadoop,dc=apache,dc=org
ldapRealm.groupObjectClass=groupofnames
# Allow userSearchAttribute to be customized
ldapRealm.userSearchAttributeName = sAMAccountName
ldapRealm.memberAttribute=member
# force usernames returned from ldap to lowercase useful for AD
ldapRealm.userLowerCase = true
# ability set searchScopes subtree (default), one, base
ldapRealm.userSearchScope = subtree;
ldapRealm.groupSearchScope = subtree;
ldapRealm.memberAttributeValueTemplate=cn={0},ou=people,dc=hadoop,dc=apache,dc=org
ldapRealm.contextFactory.systemUsername=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org
ldapRealm.contextFactory.systemPassword=S{ALIAS=ldcSystemPassword}
# enable support for nested groups using the LDAP_MATCHING_RULE_IN_CHAIN operator
ldapRealm.groupSearchEnableMatchingRuleInChain = true
# optional mapping from physical groups to logical application roles
ldapRealm.rolesByGroup = LDN_USERS: user_role, NYK_USERS: user_role, HKG_USERS: user_role, GLOBAL_ADMIN: admin_role
# optional list of roles that are allowed to authenticate. Incase not present all groups are allowed to authenticate (login).
# This changes nothing for url specific permissions that will continue to work as specified in [urls].
ldapRealm.allowedRolesForAuthentication = admin_role,user_role
ldapRealm.permissionsByRole= user_role = *:ToDoItemsJdo:*:*, *:ToDoItem:*:*; admin_role = *
securityManager.sessionManager = $sessionManager
securityManager.realms = $ldapRealm
Also instead of specifying systemPassword in clear text in shiro.ini administrator can choose to specify the same in "hadoop credential".
Create a keystore file using the hadoop credential command line:
hadoop credential create ldapRealm.systemPassword -provider jceks://file/user/zeppelin/conf/zeppelin.jceks
Add the following line in the shiro.ini file:
ldapRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/conf/zeppelin.jceks
Caution due to a bug in LDAPRealm only ldapRealm.pagingSize results will be fetched from LDAP. In big directory Trees this may cause missing Roles. Try limiting the search Scope using ldapRealm.groupSearchBase or narrow down the required Groups using ldapRealm.groupSearchFilter
PAM
PAM authentication support allows the reuse of existing authentication
moduls on the host where Zeppelin is running. On a typical system modules are configured per service for example sshd, passwd, etc. under /etc/pam.d/. You can
either reuse one of these services or create your own for Zeppelin. Activiting PAM authentication requires two parameters:
1. realm: The Shiro realm being used
2. service: The service configured under /etc/pam.d/ to be used. The name here needs to be the same as the file name under /etc/pam.d/
[main]
pamRealm=org.apache.zeppelin.realm.PamRealm
pamRealm.service=sshd
ZeppelinHub
ZeppelinHub is a service that synchronize your Apache Zeppelin notebooks and enables you to collaborate easily.
To enable login with your ZeppelinHub credential, apply the following change in conf/shiro.ini under [main] section.
### A sample for configuring ZeppelinHub Realm
zeppelinHubRealm = org.apache.zeppelin.realm.ZeppelinHubRealm
## Url of ZeppelinHub
zeppelinHubRealm.zeppelinhubUrl = https://www.zeppelinhub.com
securityManager.realms = $zeppelinHubRealm
Note: ZeppelinHub is not related to Apache Zeppelin project.
Knox SSO
KnoxSSO provides an abstraction for integrating any number of authentication systems and SSO solutions and enables participating web applications to scale to those solutions more easily. Without the token exchange capabilities offered by KnoxSSO each component UI would need to integrate with each desired solution on its own.
To enable this, apply the following change in conf/shiro.ini under [main] section.
### A sample for configuring Knox JWT Realm
knoxJwtRealm = org.apache.zeppelin.realm.jwt.KnoxJwtRealm
## Domain of Knox SSO
knoxJwtRealm.providerUrl = https://domain.example.com/
## Url for login
knoxJwtRealm.login = gateway/knoxsso/knoxauth/login.html
## Url for logout
knoxJwtRealm.logout = gateway/knoxssout/api/v1/webssout
knoxJwtRealm.redirectParam = originalUrl
knoxJwtRealm.cookieName = hadoop-jwt
knoxJwtRealm.publicKeyPath = /etc/zeppelin/conf/knox-sso.pem
# This is required if KNOX SSO is enabled, to check if "knoxJwtRealm.cookieName" cookie was expired/deleted.
authc = org.apache.zeppelin.realm.jwt.KnoxAuthenticationFilter
HTTP SPNEGO Authentication
HTTP SPNEGO (Simple and Protected GSS-API NEGOtiation) is the standard way to support Kerberos Ticket based user authentication for Web Services. Based on Apache Hadoop Auth, Zeppelin supports ability to authenticate users by accepting and validating their Kerberos Ticket.
When HTTP SPNEGO Authentication is enabled for Zeppelin, the Apache Hadoop Groups Mapping configuration will used internally to determine group membership of user who is trying to log in. Role-based access permission can be set based on groups as seen by Hadoop.
To enable this, apply the following change in conf/shiro.ini under [main] section.
krbRealm = org.apache.zeppelin.realm.kerberos.KerberosRealm
krbRealm.principal=HTTP/zeppelin.fqdn.domain.com@EXAMPLE.COM
krbRealm.keytab=/etc/security/keytabs/spnego.service.keytab
krbRealm.nameRules=DEFAULT
krbRealm.signatureSecretFile=/etc/security/http_secret
krbRealm.tokenValidity=36000
krbRealm.cookieDomain=domain.com
krbRealm.cookiePath=/
authc = org.apache.zeppelin.realm.kerberos.KerberosAuthenticationFilter
For above configuration to work, user need to do some more configurations outside Zeppelin.
1). A valid SPNEGO keytab should be available on the Zeppelin node and should be readable by 'zeppelin' user. If there is a SPNEGO keytab already available (because of other Hadoop service), it can be reused here and no need to generate a new keytab. An example of working SPNEGO keytab could be: ``` $ klist -kt /etc/security/keytabs/spnego.service.keytab Keytab name: FILE:/etc/security/keytabs/spnego.service.keytab KVNO Timestamp Principal
2 11/26/2018 16:58:38 HTTP/zeppelin.fqdn.domain.com@EXAMPLE.COM
2 11/26/2018 16:58:38 HTTP/zeppelin.fqdn.domain.com@EXAMPLE.COM
2 11/26/2018 16:58:38 HTTP/zeppelin.fqdn.domain.com@EXAMPLE.COM
2 11/26/2018 16:58:38 HTTP/zeppelin.fqdn.domain.com@EXAMPLE.COM
and the keytab permission should be: (VERY IMPORTANT to not to set this to 777 or readable by all !!!):
$ ls -l /etc/security/keytabs/spnego.service.keytab
-r--r-----. 1 root hadoop 346 Nov 26 16:58 /etc/security/keytabs/spnego.service.keytab
```
Above 'zeppelin' user happens to be member of 'hadoop' group.
2). A secret signature file must be present on Zeppelin node (readable to 'zeppelin' user). This file contains the random binary numbers which is used to sign 'hadoop.auth' cookie, generated during SPNEGO exchange. If such a file is already generated and available on the Zeppelin node, it should be used rather than generating a new file.
Commands to generate a secret signature file (if required):
dd if=/dev/urandom of=/etc/security/http_secret bs=1024 count=1
chown hdfs:hadoop /etc/security/http_secret
chmod 440 /etc/security/http_secret
Secure Cookie for Zeppelin Sessions (optional)
Zeppelin can be configured to set HttpOnly flag in the session cookie. With this configuration, Zeppelin cookies can
not be accessed via client side scripts thus preventing majority of Cross-site scripting (XSS) attacks.
To enable secure cookie support via Shiro, add the following lines in conf/shiro.ini under [main] section, after
defining a sessionManager.
cookie = org.apache.shiro.web.servlet.SimpleCookie
cookie.name = JSESSIONID
cookie.secure = true
cookie.httpOnly = true
sessionManager.sessionIdCookie = $cookie
Secure your Zeppelin information (optional)
By default, anyone who defined in [users] can share Interpreter Setting, Credential and Configuration information in Apache Zeppelin.
Sometimes you might want to hide these information for your use case.
Since Shiro provides url-based security, you can hide the information by commenting or uncommenting these below lines in conf/shiro.ini.
[urls]
/api/interpreter/** = authc, roles[admin]
/api/configurations/** = authc, roles[admin]
/api/credential/** = authc, roles[admin]
In this case, only who have admin role can see Interpreter Setting, Credential and Configuration information.
If you want to grant this permission to other users, you can change roles[ ] as you defined at [users] section.
Apply multiple roles in Shiro configuration
By default, Shiro will allow access to a URL if only user is part of "all the roles" defined like this:
[urls]
/api/interpreter/** = authc, roles[admin, role1]
Apply multiple roles or user in Shiro configuration
If there is a need that user with "any of the defined roles or user itself" should be allowed, then following Shiro configuration can be used:
[main]
anyofrolesuser = org.apache.zeppelin.utils.AnyOfRolesUserAuthorizationFilter
[urls]
/api/interpreter/** = authc, anyofrolesuser[admin, user1]
/api/configurations/** = authc, roles[admin]
/api/credential/** = authc, roles[admin]
NOTE : All of the above configurations are defined in the
conf/shiro.inifile.
FAQ
Zeppelin sever is configured as form-based authentication but is behind proxy configured as basic-authentication for example NGINX and don't want Zeppelin-Server to clear authentication headers.
Set
zeppelin.server.authorization.header.cleartofalsein zeppelin-site.xml