Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Comments on Maximize Independent Variable of Matrix Multiplication

Parent

Maximize Independent Variable of Matrix Multiplication

+3
−0

Let $T$ be an $m\times n$ matrix with column vectors $\vec{T_i}$:

$$ \vec{T_i}=\begin{bmatrix} \vec{T_1} & ... & \vec{T_n} \end{bmatrix} $$

Let $\vec{x}$ be an unknown $n$-element vector:

$$ \vec{x}=\begin{bmatrix} x_1 \\ \vdots \\ x_n \end{bmatrix} $$

Suppose the following equation holds for a known $m$-element vector $\vec{y}$:

$$ \frac{T\vec{x}}{\left|\left|T\vec{x}\right|\right|}= \frac{\vec{y}}{\left|\left|\vec{y}\right|\right|} $$

That is, $T\vec{x}$ and $\vec{y}$ have the same direction.

If each component $x_i$ of $\vec{x}$ must satisfy the condition $0\le x_i\le1$, how does one maximize the magnitude of $\vec{x}$?

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

1 comment thread

Dimensions (2 comments)
Post
+3
−0

Since all the $x_i$ are restricted to be non-negative, the magnitude of the vector increases if and only if the sum of the $x_i$ increases. Therefore, maximizing $\sum_i x_i$ is equivalent to maximizing $|x|$. This sum is a linear function.

Your direction constraint can be expressed as: $$\vec y \wedge T(\vec x) = 0\quad\text{and}\quad\vec y \cdot T(\vec x) \geq 0$$ $\wedge$ represents the outer (or exterior or wedge) product. These are both linear (in)equalities as are your constraints on $x_i$ so the whole problem is a linear programming problem. There are very well-developed tools and theories for solving these.

If you have similar problems, you may also want to look at other convex optimization forms such as convex cases of quadratic programming or second-order cone program (SOCP). If we hadn't been able to reduce the magnitude to a sum, the problem would still fall into a convex case of quadratic programming and into SOCP. In general, convex optimization is much easier and reasonably efficient to perform as compared general optimization. Unlike many areas, the dividing line between easy/efficient and hard/slow in optimization is between convex and non-convex cases rather than linear versus non-linear. (That said, linear is still nice.)

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.

2 comment threads

Linearity of Directional Constraint (2 comments)
Number of Linear Constraints (2 comments)
Number of Linear Constraints
Josh Hyatt‭ wrote almost 3 years ago

If $m$ is 2, the direction constraint simplifies to a dot product which can be represented as a single linear expression. On the other hand, if $m$ is 3, the wedge product can be replaced with a cross product, but I don't see how that could be represented with fewer than 3 linear equations. I'd guess, then, that for a given $m$, the number of constraints implied by $T\vec{x}\wedge\vec{y}=0$ is $^mc_2$. However, that's more than should be necessary to reduce the feasible region to a point for $m\ge4$, but by that logic, the number of linear constraints implied by $T\vec{x}\wedge\vec{y}=0$ should be no more than $m-1$. How many linear constraints does the direction constraint ($T\vec{x}\wedge\vec{y}=0$) imply in terms of $m$?

Derek Elkins‭ wrote almost 3 years ago · edited almost 3 years ago

Say $\vec y = \mathbf e_1$ in a set of orthogonal basis vectors $\{\mathbf e_i\}_{i=1}^m$. While there are $m \choose 2$ basis bivectors, the only ones we care about are of the form $\mathbf e_1 \wedge \mathbf e_i$ for $i \neq 1$ (as $\mathbf e_1 \wedge \mathbf e_1 = 0$), so there are $m-1$ constraints. This also makes sense because knowing the value of the dot product with $\vec y$ (assuming $\vec y \neq \mathbf 0$) would only be one linear equation, but the combination of the wedge product and the dot product would completely determine the result (because we can define the geometric product in terms of these and the geometric product is invertible for [non-zero] vectors).