Optimizely Web Experimentation Pros and Cons

What do you like best about Optimizely Web Experimentation?

My favorite thing about Optimizely is how easy the results are to understand. I'm not great with math and numbers, but with the way optimizely is set up, you don't have to calculate anything; it's either already there on the page, or you hover over it. I also like using the editor for simple changes like color or text. It's super easy to click something on the page and make changes to it. Another thing I like is how easy it is to set up pages and audiences using the tool. There are tons of options when it comes to building an audience and all the drop downs and instructions make it super easy to do. Building and grouping pages pages is another thing made easy by the tool. I typically use this feature to group similar pages together that I want to run the same test on. Most recently I grouped all of our hospital pages together so I could run an AB test on the menu of those pages. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

There are times when the website and the app you can download feel a little unstable. Sometimes the app will stop responding and go full white screen, forcing you to completely close the program and reopen it. The website can be slow at times and sometimes takes a while to load results. The results aren't always updated in real-time, like you go to a results page, and it shows the last time it updated was several hours ago, so you have to sit and wait for it to load current results. Something else I don't like is it's difficult to add new elements to the page. When I used Google Optimize, I could just drag and drop shapes and images on the page, but you can't do that with optimizely. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

The ability to test different experiences across multiple brands at the same time is fantastic. For example, we can set up a page test like if page "contains" /shop/category run these variations. This allows us to test multiple domains simultaneously under a specific type which saves us time and helps us reach Stat Sig faster in many cases. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

Optimizely is designed for a technical team. The average marketer will struggle to use the sophisticated features when trying to build experiments. Running simple changes to fonts, copy, button colors and etc. is easy enough, but they need to understand HTML, CSS and Javascript to really get full use out of Optimizely.

I was under the impression that it would be easier for a non-technical person to use, but after 6+ months of use I've learned this isn't the case. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

Optimizely experimentation platform with its newest UI update by making the process of setting up and running an experiment seamless. I appreciate the platform's integration abilities with other platforms like Segment. Also, the flexibility to send attributes in an experiment that can be used to run results segmentation is very useful during analysis. The customer success team does a great job at responding to any tickets and resolving them. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

Nothing much that I can think of except for the lack of ability to auto cleanup already existing experiments. Also, the character length limit for an event makes naming tracking events a little tricky. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

Optimizely is easy to get started with, but can also scale. It can be very detailed and advanced. Optimizely as a company is a leader in the space, and their testing strategy and methods reflect that. Their tool provides efficient, effective and well supported testing, and also resources on how to be a more effective optimizer. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

The REGEX URL fields lack definition and clarity. It would be helpful if it provided a bit more description of what was required in that field. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

Makes it easy to make tiny changes to the site - simple metrics are easy to add, can be used by non technical users. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

Requires writing workarounds to achieve anything remotely useful - if we want to implement a feature and trial it in optimizely, we have to write it _around_ optimizely. It seems the tool works against us, not with us. I feel it's overpriced for what it's doing. Still cannot upload a list of custom metrics to add and have to painstakingly do them individually. Dependence on jquery selectors is problematic for dynamic menus. Tech support terrible - unwilling to fix problems. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

Extremely simple to use interface makes it an easy-to-use platform from the outset. As a non-technical user, I can create simple experiences and add metrics that my developers have created before. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

Ultimately a developer is needed for more complex experiments or setting up goals aside from simple pageview or button clicks. We do most experiments using the development team and through them have created some complicated experiments with all the metrics we would want to track and more. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

I like the Stats engine (a very good calculation system for significance), attributes and extensions Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

From the point of view of my customer, they would like to have a quick and easy way to set up widgets. From my point of view, I can't see an overall look on A/B tests on the long term: I had the need to see which A/B test we already run on a certain page, and which have been winning and which one are inconclusive. I have also other ideas:

- have a safer way to have a cluster sampling, for instance I add before to run experiment the audience and some "subaudience" on which I want to go deep (i.e. audience: mobile, sub audience iphone X, Android phones).

- a better Documentation about recommendation implementation and a more developed feature on this (i.e. default extension for recommendation, more algorithms, a way to customize the algorithm and not only the filtering).

The sample size calculator and a "tools" area could also be included, to have always the best practices for A/B testing available. If I have something more in mind, I will reach to you. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

I've used Google Optimize for more than two years to create A/B tests and personalization experience with our Ecommerce website. Then came across Optimizely and setup up several A/B tests/personalization experiences with it as well. The unique features I noticed and seemed helpful are

The ability to pause an ongoing experiment without ending it and retaining whatever the test results up to that point until we resume or stop the experiment.

can share a preview URL of each variant in an A/B test with any co-worker or team leads without publishing the experiment to Production

Shared code feature which we can define a common Java script that is global to all the variants in a particular experiment and project JS, which is shared among all experiments in a project

minimizing the redundancy and increasing productivity

scheduling experiment

additional inbuilt support for JQuery support

integrations with other 3rd party tools and frameworks. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

Optimizely is working better in Google Chrome, and in other browsers, it crashes most of the time.

Sometimes it crashes several times in Google Chrome as well.

Several times we have observes experiments are not behaving differently in different devices in Browser stacks and real devices. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

Setting up experiments can be very easy. If you're new to website testing, Optimizely is relatively simple to configure and understand results. Documentation is also beneficial and thorough for getting started. Personalization can also be very powerful for testing different content elements for different segments. Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

Optimizely can require a highly technical skillset to get the most out of the capabilities, which can hinder a marketer with limited dev resources. Review collected by and hosted on G2.com.

What do you like best about Optimizely Web Experimentation?

The things that I like the most:

- It is very easy to setup an experiment: choose audience (who will participate), choose target pages (where you want to start you experiment) and pick the metrics (what you expect to improve), describe how you want to achieve the result (provide javascript and/or css)

- When experiment is running, usually it takes just a quick glance to understand the results

- To validate your assumptions you can always add more metrics to the launched experiment or you can filter data to show a particular user segment Review collected by and hosted on G2.com.

What do you dislike about Optimizely Web Experimentation?

Sometimes you want to know the user bucket before you render a page. For this, you need to use optimizely full-stack. Review collected by and hosted on G2.com.