Optimizely Feature Experimentation Pros and Cons

I had a chance to meet a highly talented team which gave me great insight into the CI/CD tool developed by Optomizely. The process seemed very convenient & easy to use. Review collected by and hosted on G2.com.

Some more regression tests for cutting out the edge case scenarios needs to be included to make it world class. Review collected by and hosted on G2.com.

The SDKs they offer work very well written and easy to setup, and they have support for several languages and taking application performance as something important to have in mind Review collected by and hosted on G2.com.

Not having an automatic an out-of-the-box solution for blue-green feature releases. I think this is due they are very focused on being offline, and this is not an easy problem to solve. Review collected by and hosted on G2.com.

Best part is the ease of setting up experiments and being able to change population size. Built-in conversion monitoring is also a big plus. It allows us to safely release experiments with a really quick way to role back if needed Review collected by and hosted on G2.com.

Sometimes getting the metrics in the results of the experiments to set up correctly is difficult and we do end up falling back to other tools to make sure we have an accurate picture of the results Review collected by and hosted on G2.com.

I love that it's fairly easy for me to set up ab testing and multivariate experiments with sample sizing and track for conversion success metrics. Review collected by and hosted on G2.com.

3 painpoints:

1) Hard to use on mobile device

2) UI is not as great as web

3) Limited configurability via code which is often a win for Devs Review collected by and hosted on G2.com.

Easy to alter variables and tag up, easy to start and pause tests. Can view results clearly. Review collected by and hosted on G2.com.

I feel like it could have a better modernised UX as it's looked the same for a while. Review collected by and hosted on G2.com.

The Optimizely Full Stack tool provides the capability to learn by using data and controlling the UI and the data that powers the user experience. We can leverage our designers and engineers to develop tests that move the needle on guest satisfaction while maximizing revenue. By partnering with the Optimizely team, our engineers were able to build a containerized version of Full Stack and allow more than 22 product teams to harness the goodness of this tool via API. The partnership involved was the best I've experienced in my 20+ years of software development. Review collected by and hosted on G2.com.

One area that has created some concern for our teams is how easy it is to publish winners to production 100%. We have had cases where business folks that lack the complete understanding of existing user flows that could be impacted by promoting a test have just promoted the winner without consulting the product engineering team. The ease of promoting winners is a good feature, but there must be governance that needs to be behind it to ensure the right experiences go into production. Review collected by and hosted on G2.com.

The flexibility and ease of use when you know exactly what you want to do with the platform and your site. From Multivariate tests to redirects, re-arranging content on pages or making changes to multiple pages all in one experience.. Optimizely has been able to do it all for my team. Review collected by and hosted on G2.com.

I wouldn't say there's much I dislike with Optimizely as much as there were some limitations in using the platform when it comes to how my companies website was structured. So with that being said, make sure that you will be able to achieve everything you want using the tool based on the setup of your web property. For reference, I was using Gatsby+Contentful and was unable to make specific changes to modules on pages that shared the same class in the code. Review collected by and hosted on G2.com.

I am a huge fan of the freedom and flexibility feature experiments functionality provides to the end-user. I have saved loads of time and money by utilizing this feature and have been able to run multiple multivariant A/B tests across my product with just the flip of a switch. I can quickly customize/differentiate my features in real-time based on data trends and can reduce the risk of large feature roll outs through the use of the feature flag functionality. Review collected by and hosted on G2.com.

As is the case with most tools, one of the pain points with Optimizely is that it still requires developer support and does not always work seamlessly out of the box. We have not been able to fully utilize the stat sig calculations Optimizely provides because there are inconsistencies between this tool and our BI tool, which can be complicated to work through. Review collected by and hosted on G2.com.

The dashboard is clean, seems organized. Ability to create feature tests, split tests, and run variables for different cohorts. Review collected by and hosted on G2.com.

Completely unusable on mobile devices. If there was a sudden need to change a configuration or disable a test in an environment altogether, you can't do so while on the go. Additionally, there is no way to change a feature test into a simpler a/b test or vice versa if the experiment design needed to change course. Review collected by and hosted on G2.com.

Development is the best for stack considering all point's Review collected by and hosted on G2.com.

It should be more good in technology and it should also be helpful to aware peoples Review collected by and hosted on G2.com.