Toss chicken, potatoes, bell peppers, onion, and garlic in a large bowl with olive oil. Spread onto a large sheet pan or jelly roll pan in a single layer. Squeeze lemon wedges over the top, then place the spent wedges on the sheet pan. Sprinkle oregano over everything. Season with salt and pepper. Add Kalamata olives.