There are a few ways to get around in The Elder Scrolls Online: Tamriel Unlimited. The first of which is to use your legs and walk. Jokes aside, that's pretty much the primary method of getting around. There are two alternate methods, one of which is the horse which provides a minor speed increase until you level it up and the other is the wayshrine system.

Horses

We have a guide to mounts here that goes into vast detail about horses in TESO, but in general horses are slow and must be evolved over time. You can get a horse rather early in your adventures, but that doesn't mean it's going to be going rather fast. So I consider horses to be the primary mode of transit, but one of the slowest for the longest time.

Sprinting

By pressing down the SHIFT key by default, you can spring. Which consumes stamina. This is the easiest way to move quickly through the world, but it does put you in a weird disadvantage. First, it consumes stamina so if you have to battle you'll need to combat regen your stamina in order to fight with stamina abilities. Next, stamina only has a finite amount, so once you run out you quit sprinting.

My biggest suggestion is when you're almost out of stamina to let go of the button. If you keep holding it down, you won't regen stamina and will continue to walk at a normal pace.

Wayshrines

Wayshrines are littered throughout the world and work in many different ways. The shrines offer a resurrection point if you don't have a soul shard on you. You'll resurrect at the nearest wayshrine after death and from there can continue your journey.

The next use is as a transit station. You can go to a wayshrine and teleport directly to any other wayshrine for FREE. That's right, no cost, nothing. You just have to discover the two wayshrines and you get a free ride. Isn't that crazy?

The final use of a wayshrine is that they can be teleported to while out in the world for a fee. I suggest only doing this is you're rich and lazy, since there is enough of them that it takes all of 30 seconds usually to find one to walk to. You can use the map to interface with remote shrines if you do wish to pay the fee.

Transitus Shrines

In PvP the wayshrines become "Transitus Shrines" which allow you to move from one keep to another. It's actually a network of nodes, if there are two connections to a node available then you can instantly teleport to that node. However, if one of the nodes is under attack, you can't teleport.

So essentially if there are two sister keeps captured and not under siege, then you can teleport to the location. This is part of PvP tactics because controlling which nodes are operational and which ones aren't is critical to being able to control the flow of enemies.

Other Methods

Quests can open portals that will allow you to transfer to different areas and different instances. Generally speaking, a quest will teleport you to an area that is instanced and blocked off, and after the quest completes you'll be done.

You can also travel to PvP through the AvA menu. In the PvP maps you can travel to the wayshrines for a quick exit to the nearby PvE maps.

That's about it for traveling in ESO!