Skip to content

add support for tile - #317

Closed
Bahaatbb wants to merge 5 commits into
ml-explore:mainfrom
Bahaatbb:tile-as-op
Closed

Bahaatbb wants to merge 5 commits into
ml-explore:mainfrom
Bahaatbb:tile-as-op

Conversation

@Bahaatbb

Copy link
Copy Markdown
Contributor

Proposed changes

np.tile function in mlx as a op mlx.core.tile

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

@angeloskath angeloskath left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this. However, tile should be implemented using broadcast_to and reshape rather than repeat to avoid doing multiple unnecessary copies.

See #318 for an improved repeat implementation that could be used as a guide for a tile.

@Bahaatbb

Bahaatbb commented Dec 31, 2023 •

Copy link
Copy Markdown
Contributor Author

@angeloskath Hey, Happy new year 🎉.
sorry if my first implementation is not up for the standards, tho I must say that it is almost the same implementation as numpy.
tho, I went back and tried this implementation and hacked together this code using your idea, the below code is just a prototype, to take your approval to proceed.

def tile(arr, repeats):
    d = len(repeats)
    
    if d < arr.ndim:
        repeats = [1] * (arr.ndim - d) + repeats
    else:
        arr = np.expand_dims(arr, axis=tuple(range(d - arr.ndim)))

    for i, rep in enumerate(repeats):
        if rep != 1:
            arr = np.expand_dims(arr, axis=i)
            arr = np.broadcast_to(arr, arr.shape[:i] + (rep,) + arr.shape[i+1:])
            arr = np.reshape(arr, arr.shape[:i] + (-1,) + arr.shape[i+2:])
            
    return arr

Note: I change the implementation code to match python code.

@angeloskath angeloskath left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for being late to review it. I left some comments, I think after these it should be mergeable. Thanks!

Comment thread mlx/ops.cpp
}

int d = repeats.size();
array arr = copy(a, s);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy is not needed here, you can simply do array arr = a;.

Comment thread mlx/ops.cpp
new_shape.erase(new_shape.begin() + i + 1);
arr = reshape(arr, new_shape, s);
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already much better than before, however again each of these reshapes will be an actual copy and there is no need to launch many. Basically this is like calling repeat() many times.

The better way to do it is to keep 3 vectors.

  • a reshape vector which will do all the expands at once (start with a.shape() and add 1s where needed)
  • a broadcast to vector which will have the number of repeats where 1s were added in the above
  • final reshape vector where the repeats are merged to the axis to their left

After that the implementation is as simple as the following:

auto out = reshape(a, expand_shape, s);
out = broadcast_to(out, repeat_shape, s);
out = reshape(out, final_shape, s);

Comment thread mlx/ops.cpp
for (size_t i = 0; i < repeats.size(); i++) {
if (repeats[i] < 0) {
throw std::invalid_argument(
"[tile] Negative repeat value found at index " + std::to_string(i));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extremely nitpicky, feel free to ignore. Using an ostringstream would be better as it would allow you to also provide the whole repeats vector. Something like:

std::ostringstream msg;
msg << "[tile] Repeats argument cannot contain negative values but "
    << repeats << " was provided.";
throw std::invalid_argument(msg.str());

@dc-dc-dc

Copy link
Copy Markdown
Contributor

is this pr stale ?

@dc-dc-dc dc-dc-dc mentioned this pull request Jan 12, 2024
4 tasks done
@Bahaatbb Bahaatbb closed this Jan 12, 2024
@Bahaatbb

Bahaatbb commented Jan 12, 2024 •

Copy link
Copy Markdown
Contributor Author

Sorry, I haven't had the chance to get back to this, I think did what is required @dc-dc-dc .
I will close this and look for more in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants