There are a few important differences between any definition of a ternary operator in any language, and an if statement.
The main one is that the latter is a statement, whereas a ternary operator is by definition an expression. In other words, you expect it to return a value. An if block in octave/matlab does not "return" a value (i.e. you can't do a = if ... endif). If you need something 'returned', you need to assign it to an external variable inside the loop, and then use that variable outside the loop.
The second big thing, is that a ternary operator should be chainable. Ie you should be able to do "if this, then that, otherwise if that then that other thing", such that at the end of a long chain of comparisons you return a single value.
Now, octave and matlab do not have this kind of operator. But in octave, it's easy enough to simulate one, e.g. with cells:
M1 = false; M2 = false; M3 = true;
{'expression1', 'expression2', 'expression3', 'fallback'}([ M1, M2, M3, true ]){1}
If you really want to, you can make that kind of thing into a nice anonymous function, which returns the first expression for which its mask/test was true:
tern = @(varargin) reshape(varargin,2,[])(2,:)([reshape(varargin,2,[]){1,:}]){1}
tern( M1, 1, M2, 2, M3, 3, true, 4 ) % ans = 3
where 'fallback' is achieved simply by having a test that evaluates to 'true' explicitly before the 'fallback' value.
Note: This style of 'chaining' two or more indexing operations inside an anonymous function only works in octave, as unfortunately matlab does not allow chained operations, e.g. a(1)(2). There is nothing stopping you from making an equivalent 'proper', external function though, where you save intermediate results to a temporary variable before indexing it a second time; therefore, conceptually, this strategy will also work in matlab.
Alternatively, you could make a simpler 'ternary operator' (well, function) using a slightly different syntax, where you pass a 'tests/masks' array, and 'expressions', and 'fallback' cells separately. E.g.
tern2 = @(M,E,F) [E, F]{find([M, true], 1)}
tern2( [M1, M2, M3], {1, 2, 3}, {4} ) % ans = 3
Finally, there is also the ifelse function, which is similar in spirit, i.e. ifelse(test, expr_if_true, expr_if_false), but this is not really a true ternary operator/function, since it is not chainable; it's more useful for choosing between two alternatives based on a 'true/false mask', e.g.
ifelse( [M1, M2, M3], {1, 2, 3}, {4,5,6} )
% ans = { 4, 5, 3 }
