Term

Definition

A/B Testing

A/B testing is a type of test in which your users are separated into groups (variants) and each group is served a different experience. The control group is served the default experience while the variant(s) is served the alternate experience. The variant(s) is then compared against the control with metrics/goals to establish the winner.

A/B Testing Goals

The goals are success measurements used to determine the winning variant of the test. The individual goals are used for the time estimate for the test to reach statistical significance. Any ratio or percent available in the metric catalog can be used as a goal.

Action(s)

Action has the meaning of API Method as defined below, and includes each individual piece of session, user, event, or state data communicated to Leanplum through API Calls to the Leanplum Service.

Active Mode

Active status is when an A/B test or messaging campaign has been created and published live to the targeted users.

Admin Dashboard

The Admin Dashboard allows you to manage your apps, users, and test devices. Leanplum provides user permission settings to ensure that users in your organization have access to the appropriate apps and abilities within your account.

API Call(s)

API Call(s) has the meaning located at https://www.leanplum.com/docs/api, as may be amended by Leanplum from time to time in its sole discretion without prior notice. API Calls, which are generated without using the SDK Materials, may in some instances also be referred to as “Server-side API Calls.”

API Method

The mechanism through which a subscriber of the Leanplum Service may communicate with respect to an individual piece of session, event or state data to Leanplum through API calls directed to the Leanplum Service. For the avoidance of doubt, an API call, as defined above, may contain multiple API Methods.

App ID

This is the ID applied to your application in Leanplum that tells us which dashboard the data is applied to.This key is used in your Leanplum integration.

App Settings Page

The App Settings page allows you to create and delete apps, and adjust all app settings. An app will be tied to whichever team you create it under, and all team members from that team will have access to it.

Calendar UI

The calendar UI provides a comprehensive view of scheduled push notifications on a monthly calendar. The UI provides visibility into how often users are being communicated with and which messages are being sent.

Content Read-Only Key

This key is used when retrieving information about your tests and messages from Leanplum with our API. It is specific to your app in the Leanplum dashboard. See our available content read-only methods.

Control Group

The control variant is what will be used to compare your result against in the A/B test. If testing a variable, the control will be the override value on the Variables Dashboard. If no override has been set, it defaults to the default in the code value.

Data Export Key

This key is used when exporting data from Leanplum with our API. It is specific to your app in the Leanplum dashboard. See our available data export methods.

Development Key

This key is used to tell leanplum to send the data to the development/test pipeline. This allows the data to be processed real time as well as to viewed in the debugger for validation. It is specific to your app in the Leanplum dashboard. See our available development methods.

Distributor Toggle

The distribution toggle allows you control the portion of users randomly assigned to each variant in the A/B test.

Draft Mode

Draft status is when an A/B test or messaging campaign has been created but not published live to users

Element Selector

For the UI editor, this allows you to select any element from your interface here. The selected element is highlighted in purple.

Email Message

A server-generated email message sent from the Leanplum Service to an email recipient at the direction of the Subscriber.

Finished Mode

Finished status is when an A/B test or messaging campaign has been published and completed.

First Time User

A first time user is a user who has installed the app and opened it (initializing Leanplum) for the first time. This is defined in Leanplum as users with 0 sessions prior. If the user installs the app but does not open it, they will not count towards the first time user analytic.

Immediate Sending

A delivery type that sends messages to users within the next several minutes.

Impression Criteria

The impression criteria defines when a user is counted towards the analytics of the test. By setting an event as the impression criteria, the user will not count in the analytics of the test until they fire the event selected.

In-app Messages

A type of message displayed to its targeted users at a chosen place within the app.

Lifecycle Engine

The lifecycle engine is used to create an automated messaging campaign that sends users a series of messages based on past behavior. A trigger/event is defined that places the user in the campaign as well as a trigger/event is defined that takes the user out of the campaign, i.e. the campaign objective.

Magnet

The magnet icon controls the “stickiness” of your test. As a default, the magnet is turned on and lit blue. This means that even if a user no longer matches the initial targeting criteria, they will continue to be part of the test. You can turn the icon off, which will turn the button grey and will remove users from a test if they no longer match the targeting criteria.

Manual Sending

A type of delivery that allows apps to send messages using the REST API.

MAU or Monthly Active Users

The greatest number of unique users seen in all of Subscriber’s Application(s) for the applicable calendar month.

Messages

A message means Push Notification, Email Message, or Webhook Call.

Message Actions

Performs an in-app action that may not display a message, such as sending the user to a specific url (in-app or external)

Metric Catalog

The metric catalog appears when you click the + tile in analytics or attempt to add a goal to an A/B test. These are all the metrics that are currently being tracked for the application and can be selected for analytics. See the Metric calculation glossary for more.

Newsfeed

A type of message that sends to an inbox contained within your app that Leanplum manages for your users to revisit.

Paused Mode

Paused status is when an A/B test or messaging campaign has been published live to the targeted users but subsequently paused.

Premium Support

Customer Success
Leanplum will provide a named Customer Success Manager overseeing the engagement. This includes regular QBRs including executive involvement, quarterly roadmap previews, continuous training for new users, and bi-monthly meetings. An escalation path is provided for Platform/Technical support and contractual escalations:

Platform/technical support: On call support (via [email protected]) -> Assigned Account Manager -> Global Lead CS -> VP CS & Serivces
Legal & Contract support: Growth Account Manager -> Renewals Team Lead -> VP CS & Serivces -> CRO

Technical Support
Leanplum will provide premium technical support with a response rate <= 4 hrs (see detailed SLA found at https://www.leanplum.com/service-level-agreement), access to the online knowledge base and case portal. In addition, direct access to senior support engineers and quarterly support reviews will be provided. An escalation path is provided for Platform/Technical support and contractual escalations:

Platform/technical support: On call support (via [email protected]) -> Assigned Account Manager -> Global Lead CS -> VP CS & Serivces
Legal & Contract support: Growth Account Manager -> Renewals Team Lead -> VP CS & Serivces -> CRO

Preview Panel

The preview panel will display the message on the viewer to the right side as one would see it on their device.

Production Key

This key is used to tell leanplum to send the data to the production/live pipeline. It is specific to your app and Leanplum dashboard. See our available production methods.

Professional Services

Services provided by Leanplum to assist Subscriber with its use of the Leanplum Service, such as implementation, configuration and training services. The specific services to be provided will be as specified in the applicable Service Order. Professional Services are not included within the scope of “Leanplum Service” or “technical support services”.

Push Notifications

A server-generated message sent from the Leanplum Service to Subscriber Users’ mobile devices, which is delivered without a specific request from such mobile devices. Push Notifications are sent though platform-dependent services such as the Apple Push Notification service (APNs) or the Google Cloud Messaging (GCM) service. Leanplum can add and remove integrations with platform-dependent Push Notification services from time to time in its sole discretion without prior notice.

Returning User

A returning user is a user who has installed the app and has had at least 1 session prior. On the day that they are viewed as returning user, it will be their 2nd or greater session.

Scheduled Sending

A delivery type that results in users receiving the message on the day and time option selected. There are several time options that Leanplum provides.

Standard Support

Customer Success
Leanplum will provide standard account management oversight which includes a regular business review, product updates, access to the customer newsletter & the Leanplum academy. Training will be provided on a quarterly basis post activation as needed. An escalation path is provided for Platform/Technical support and contractual escalations:

Platform/technical support: On call support (via [email protected]) -> Assigned Account Manager -> Global Lead CS -> VP CS & Serivces
Legal & Contract support: Growth Account Manager -> Renewals Team Lead -> VP CS & Serivces -> CRO

Technical Support
Leanplum will provide standard technical support with a response rate <= 1 Business day (see detailed SLA found at https://www.leanplum.com/service-level-agreement), access to the online knowledge base and case portal. An escalation path is provided for Platform/Technical support and contractual escalations:

Platform/technical support: On call support (via [email protected]) -> Assigned Account Manager -> Global Lead CS -> VP CS & Serivces
Legal & Contract support: Growth Account Manager -> Renewals Team Lead -> VP CS & Serivces -> CRO

Statistical Significance

Significance represents the likelihood that when Leanplum identifies a winning variant, it is in fact a winning variant. If you choose a higher significance, you reduce the risk of a false positive (i.e. concluding your goal was reached when in actually was not), but your experiment will need to run for more time. Our defaults of 80 percent power and 95 percent significance works well for most applications, and we recommend you only change them if requested by your data science team. We calculate significance with the Welch T-test.

Status Indicator

This icon tells you whether a message is active (green), inactive (gray), paused (dotted circle), or finished (check mark). A/B tests also show a status indicator icon, but with only active (green) or inactive (gray) statuses.

Targets

Targets allows you to select groups of users based on attributes about them or actions they have taken. You can use targets in A/B tests and/or messaging campaigns to focus on specific users. See all Segment and target options for more info.

Team Management page

The Team Management page allows you to edit your account settings, create, edit, or delete teams, and add or remove team members. All apps in Leanplum belong to a team, and all users that are members of a team have access to all apps that belong to that team.

Test Devices Management page

The Test Devices Management page allows you to manage which devices are registered as development devices. To add a development device, either run a development build of your app on your device, or find an existing user via the Users dashboard and click Register next to the device.

Time Estimator

The Time Estimator gives an approximation of when the test will reach statistical significance. If the test is estimated to take a long time to complete, you can alter several things in your test to help minimize the time estimated. The key things that impact the test time estimation are: 1. The goals selected 2. The targets selected 3. The number of variants in the test.

Triggered Sending

A delivery type in which users must complete the selected action in order to trigger the push notification to send at the desired time delay after said action has been completed.

Upgraded Support

Customer Success
Leanplum will provide a named Customer Success Manager overseeing the engagement. This includes regular QBRs including executive involvement, quarterly roadmap previews, continuous training for new users, and bi-monthly meetings. An escalation path is provided for Platform/Technical support and contractual escalations:

Platform/technical support: On call support (via [email protected]) -> Assigned Account Manager -> Global Lead CS -> VP CS & Serivces
Legal & Contract support: Growth Account Manager -> Renewals Team Lead -> VP CS & Serivces -> CRO

Technical Support
Leanplum will provide premium technical support with a response rate <= 4 hrs (see detailed SLA found at https://www.leanplum.com/service-level-agreement), access to the online knowledge base and case portal. In addition, direct access to senior support engineers and quarterly support reviews will be provided. An escalation path is provided for Platform/Technical support and contractual escalations:

Platform/technical support: On call support (via [email protected]) -> Assigned Account Manager -> Global Lead CS -> VP CS & Serivces
Legal & Contract support: Growth Account Manager -> Renewals Team Lead -> VP CS & Serivces -> CRO

User bucket

A randomly generated number between 0 and 999 assigned to each individual user. User buckets are the easiest way to randomly target a portion of your users with a message or A/B test.

User Operations

A User Operation is counted each time a user profile is accessed by the Leanplum Service. User profiles can be accessed through SDK calls, server- side API calls, third-party integrations, or messaging — all of these count as a User Operation. Online documentation is available for review at https://docs.leanplum.com/docs/billing-definitions

Variable

A variable is anything in the app that you may want to swap in and out on the fly or A/B test. This can be a line of text, an image, a layout file or a new feature. Once a variable is coded in and sync’d on Leanplum, you can change the default as you like without having to resubmit to the app store.

Variable Default in Code

This is the default that the user will see should no change be made to the variable. The value listed here is what the developer coded in as the default.

Variable Overrides

The variable override is where you input the change in value/experience that you want the users to see as opposed to the default.

Variant

The variant is the change that you are making in the test and comparing with that of the control. Simply click on the value in the variant and change it to the desired option that you want served to your users in the test.

Webhook

A type of message that sends through another service, such as an email provider.

Webhook Call

An API call initiated from the Leanplum Service to Subscriber’s API service or a third-party API service, which is not a Push Notification or an Email Message, and is not initiated for the purpose of obtaining external data for composing a message within the Leanplum Service.


Did this page help you?