A ritual is the act of casting a spell or using a spell-like or supernatural ability which requires a very long time, much longer than could possibly be used in a combat situation.
The exact time required for a ritual is typically measured in hours, with the specific casting time described in the spell or ability write-up.
Unlike many spells with a casting time, some rituals have their magical effect occur during the casting time for the ritual. An Alter Winds spell, for example, may be cast as a ritual, and the effect (increased overland movement for vehicles or beings using wind) occurs during the casting time. A Teleport spell, however, can be cast as a ritual, and in that case there is no effect until the full duration of the casting time is reached, when all of the effect of the spell is released at once.
If the casting creature does anything at all during the casting of a ritual spell other than continue the ritual, the spell is lost. If the ritual had an effect during casting, that portion of the effect which had already been gained remains. If the ritual had a single effect at the end of the casting, then that effect is completely lost. For example, if an Alter Winds was being cast on a sailing ship, and after four hours a combat happens and the character casting Alter Winds participates in the battle in any way, then the ship gains four hours of increased movement and the spell ends. If a Teleport spell was being cast for eight hours to reach a distant objective, and after four hours a combat happens and the character casting Teleport participates in the battle in any way, then all benefit of the Teleport is lost and the spell ends.
A character casting a ritual may make concentration checks as normal to maintain their ritual. If a creature attempts to 'concentrate through' a combat, one concentration check is made each round if they do nothing to participate in the combat. If they are drawn into the battle directly (if they are attack, for example) they must make concentration checks as normal for casting any spell, in addition to the one per round to keep the ritual going.
In all cases, refer to the exact writeup of the ritual spell for details of how each individual spell is handled.