What is the Project Format?

Troubleshooting

How to View Project Format Reports

In the Project Format test results report, you can gain a multi-faceted view of a candidate's coding skills and team development capabilities. This page explains the meaning of each section in the report and how to review submitted code and Pull Requests (PRs) on GitHub.

Opening the Report Tab

Click the "Report" tab at the top of the candidate details page to display the report screen.

Summary Section

Total Score

The pie chart in the upper left of the screen displays the overall score and rank across all problems.

Scores by Problem

A list of scores and star ratings (on a 3-level scale) for each problem is displayed. If there are multiple problems, you can check the weight of each problem in the points distribution graph.

Relative Evaluation

You can check the percentile rank compared to your own company's past test-takers or the overall population. Filtering is available by population, aggregation period, and tags. If the number of test-takers does not meet a certain threshold, "Insufficient data" will be displayed.

Problem Details Section

A "Project" badge is displayed next to the problem name.

Score Breakdown

In Project Format problems, scores are divided into different evaluation axes.

Evaluation Axis Description
Coding Skills Evaluates code quality, such as reusability, naming conventions, and readability.
Team Development Skills Evaluates the development process, such as commit size, commit messages, and Pull Request descriptions.

Each evaluation item is displayed as ✅ (Pass) / ❌ (Fail). You can check the details of the grading reasons by clicking the "Show all details" toggle in the upper right.

Submission Details

Item Description
Elapsed Time The time the candidate spent on the test.
Commits The total number of commits up to submission.
Total Additions The number of lines of code added.
Total Deletions The number of lines of code deleted.

Reviewing Submitted Code

Open Submitted Code

Click the "Open Submitted Code" button to view the submitted code in the HireRoo code viewer.

Review on GitHub

Click the "Open PR (GitHub)" button to view the Pull Request submitted by the candidate directly on GitHub.

Note: Access rights to the repository are required to view the GitHub PR.

Requesting GitHub Access Rights

Application Procedure

  1. Click the "Request Access" button.

  2. A modal will open; enter your GitHub Account ID.

  3. Click "Next" to see a confirmation screen. Verify the account ID to be granted access and click "Submit Request".

  4. An invitation email will be sent to your GitHub email (or via GitHub notifications). Click "Accept invitation" to approve joining the repository.

  5. When you return to the HireRoo report screen, the requested account will be added to the "Accounts with access" section. You can now view the PR.

Important Notes

  • The access period is 7 days from the date of the request.
  • Please use the access rights for review purposes only.

Points to Deepen in the Interview (AI Generated)

In the "Points to Deepen in the Interview" section at the bottom of the report screen, AI automatically generates sample interview questions based on the submitted code.

Generation takes about 3 to 4 minutes. "Loading" may be displayed immediately after opening the screen; please wait a moment.

The generated interview points can be used in the next interview step. By clicking the "Create Interview from Report" button, you can start an interview based on this report immediately.

Candidate View Review Screen

Candidates who have been granted access by the hiring manager can access app.hireroo.io/e/take-homes/{ID}/review to check their own submission details.

  • Left Pane: Problem statement, functional requirements, and API specifications.
  • Center Pane: Tree view of submitted files.
  • Right Pane: Diff of the selected file (added and deleted lines are highlighted).

An "Archived" badge is displayed for problems where the test has ended.

Frequently Asked Questions

Q. The relative evaluation graph is not displayed. A. It will be displayed once the number of test-takers in the population exceeds a certain number. If the number of test-takers is low, "Insufficient data" will be displayed.

Q. I cannot open the GitHub PR. A. Please register your GitHub Account ID via "Request Access" and accept the GitHub invitation. If the access period (7 days) has passed, you will need to re-apply.

Q. "Points to Deepen in the Interview" is not generating. A. Generation can take up to 3-4 minutes. If it does not appear after some time, please try reloading the page.