Optimizely Web Experimentation Pros and Cons

Both the speed with which you can undertake experiements and changes, meaning you can validate lots of hypothesises in a short space of time, and the ability to have reassuances on the data impacts (and calculation of statisitcal significance). The WYSIWYG editor significant helps team members with low technical accumen. Have had great help with Optimizelys customer support team. Review collected by and hosted on G2.com.

Dependent on the change, your tech stack needs to be able to work with Optimizely. Some of the tests that you feel could easily be accomplished in Optimizely Web, prove to be more difficult, and there is a reasonable learning curve to understand what can be feasibly achieve in the platform and what cannot be. Review collected by and hosted on G2.com.

I love that it allows us to track specific types of conversions that we can define. It has clear reporting that shows the winning variant and works well with relatively minimal setup. Review collected by and hosted on G2.com.

I felt that there was more javascript development needed to conduct multi-page tests and more advanced modifications. The tool did not like us making tons of changes within the HTML and preferred that we made those modifications with JS. Review collected by and hosted on G2.com.

I've been using Optimizely Web Experimentation for a couple of years. It can handle just about any experiment we throw at it, has a robust reporting engine, they introduce new features at a fast rate and the performance is good. It's an efficient platform to use as well. The customer support team has also been very helpful whenever we've needed them. Review collected by and hosted on G2.com.

While mostly easy to use, you do need a basic understanding of CSS selectors and the DOM. Without that base level you'll find it a frustrating experience trying to select the right element to change or measure. Review collected by and hosted on G2.com.

The ease of implementing an experiemnt with the WYZYWIG editor was very easy to get to grips with. The logical sequence and selection of options to setup the experiments are also easy to get to grip with.

The onboarding and customer team have been very helpful with getting us setup.

Installation of the tracking script also took minutes rather than a deep technical installation, meaning the integration was simple. Review collected by and hosted on G2.com.

Setting up metrics is an aspect which needs time and attention- if these haven't been setup correctly then the results panel will not be helpful when reviewing results. Review collected by and hosted on G2.com.

The live editor allowing you make customizations on the fly is an exceptional tool. Its easy to implement, use, and unlike anything we've used before. It really put the power in the right hands and decouples the engineering dependencies from the experimentation process. Also, the edge capabilities are really powerful in delivering an optimized experience for our customers. Review collected by and hosted on G2.com.

The downside we've seen is from a testing perspective and having an ability to force yourself into one of the variations. It would be great for our QA team to be able to decide which version they want and ultimately be able to test any version at their discretion. Review collected by and hosted on G2.com.

The upsides are that you can easily put experiments live. I do not have CRO experience and putting these experiments live was very straightforward. The tracking of results and the statistical significance is also very clear. We like how we can also test into different pages on site not just as a whole which a different competitor we used to use did which limited us Review collected by and hosted on G2.com.

One thing I’ve found is that I can’t check revenue per visitor in the dashboard only revenue per session which limits our reporting and need to go to GA for this. Review collected by and hosted on G2.com.

- Ease of use

- Strong enablement for wide variety of practicioners specifically via the Visual Editor

- Fantastic response time from support

- Supersped our CRO program

- Easy onboarding across wide range of projects from an implementation POV

- Easy to verify and read metrics to present to range of stakeholders. Enables decisioning process from a much speedier POV Review collected by and hosted on G2.com.

- Some challenges with cross-domain tracking of EndUserId but that's quite a specific use case and isn't representative of overall ease of implementation Review collected by and hosted on G2.com.

There are a wide range of tests that I can run using the platform: A/B, A/B/n, multivariate, multi-arm bandit and 'stats accelerators'.

The trigger options allow me to quite easily target discreet customer segments by device type, or browser type, or by another specific behaviour. I've also done tests using custom JavaScript to suppress tests for more sensitive customer segments.

I like the reporting dashboard, which is good for conducting dip checks on how specific tests are performing across different browser and device types - and for seeing how a test is performing over time.

I also like the fact I can set a test to roll out to 5% of traffic, conduct a few post-live checks and then quickly duplicate the test and roll it out to 100% of eligible traffic.

Customer support tickets are via email and I usually receive a response within a few hours, which is good. Review collected by and hosted on G2.com.

The visual page building tool is quite basic and could do with a refresh. It means that you more or less have to know HTML and CSS to build your own simple tests, and JavaScript for anything more complicated.

Many CRO practitioners will use Bayesian probability alongside Frequentist statistics if traffic volumes are smaller for specific tests. The reporting dashboard defaults to statistical significance - so Bayesian needs inclusion. Review collected by and hosted on G2.com.

There are plenty of configuration options, which helps ensure we're running experiments under the right circumstances, or for the correct user groups. Review collected by and hosted on G2.com.

It was difficult to implement, and it's rather sensitive to different technical environments. This won't be immediately apparent. It could result in extended deployment times and many calls with various Optimizely teams. Review collected by and hosted on G2.com.

Optimizely keeps growing and maturing the feature functionality. Integrations with my backend, code snipits to go in GTM and willingless to leave behind a plan for people to follow inside the org is helpful. Review collected by and hosted on G2.com.

The hardest part is going cold pre testing and personalization into a culture where folks self serve. That is the stage my organization is at right now and while I am confident there is support to help us get there, the challenge to explain the value of testing remains. Review collected by and hosted on G2.com.