How to check for an operator property

Question

Trying to modify this to show up in the keymap instead of in toolsettings https://developer.blender.org/D6322

Creator says it was done as a tool setting so it can know when to draw facedots in xray. Don’t really care about that either way. Can live without it, but I am pretty sure if I get an understanding of my problem it would allow this functionality.

What I’m interested in is getting this in the keymap, so it can be mapped to a modifier similar to the different selection modes.

I figured out how to get another checkbox in keymap, and where I want it to be, but I don’t know how to make a script check if that property is true/false, the same way it is currently checking if a tool setting is true/false.

Here is my new property ‘mesh_select_through’ in wm_operator_props.c

void WM_operator_properties_border(wmOperatorType *ot)
{
  PropertyRNA *prop;

  prop = RNA_def_int(ot->srna, "xmin", 0, INT_MIN, INT_MAX, "X Min", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_int(ot->srna, "xmax", 0, INT_MIN, INT_MAX, "X Max", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_int(ot->srna, "ymin", 0, INT_MIN, INT_MAX, "Y Min", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_int(ot->srna, "ymax", 0, INT_MIN, INT_MAX, "Y Max", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);

  prop = RNA_def_boolean(ot->srna, "wait_for_input", true, "Wait for Input", "");
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_boolean(ot->srna, "mesh_select_through", true, "Select Through", "");
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
}

It shows up in the keymap under ‘Wait for Input’ which is what I was hoping for. Now I just need the script to check for that property instead of its tool setting equivalent. This is done in two scripts from what I can tell, and getting it to work in one should be the same process as in the other.

I think I probably have to add an Include to something in that script, and then set a *prop variable to it? That’s the way it is checking for the tool setting, so it should work the same way, right?

Here’s what it does to check the ‘mesh_select_through’ tool setting and then do something about facedots in view3d_select.c

static bool do_mesh_box_select(ViewContext *vc,
                               wmGenericUserData *wm_userdata,
                               const rcti *rect,
                               const eSelectOp sel_op)
{
#ifdef DEBUG_TIME
  double t1 = PIL_check_seconds_timer();
#endif
  BoxSelectUserData data;
  ToolSettings *ts = vc->scene->toolsettings;

  view3d_userdata_boxselect_init(&data, vc, rect, sel_op);

  if (SEL_OP_USE_PRE_DESELECT(sel_op)) {
    if (vc->em->bm->totvertsel) {
      EDBM_flag_disable_all(vc->em, BM_ELEM_SELECT);
      data.is_changed = true;
    }
  }

  /*for non zbuf projections, dont change the GL state */
  ED_view3d_init_mats_rv3d(vc->obedit, vc->rv3d);

 GPU_matrix_set(vc->rv3d->viewmat);

  const bool use_zbuf = !(ts->mesh_select_through || XRAY_FLAG_ENABLED(vc->v3d));
  const bool show_face_dots = (vc->v3d->overlay.edit_flag & V3D_OVERLAY_EDIT_FACE_DOT) != 0;

It does a bunch of other stuff in view3d_select.c to make it all work, and something else in another script to draw facedots or not.

So I’m thinking all I have to do is set this up to check my operator property instead of the tool setting. But I don’t know how, and looking around has gotten me nothing, sorry.

edit: It occured to me to check wm_operators.c to see how it makes use of properties, so I did this:

  BoxSelectUserData data;
  ToolSettings *ts = vc->scene->toolsettings;
  wmOperator *op;
  PointerRNA *ptr;
  PropertyRNA *mesh_select_through = RNA_struct_find_property(op->ptr,
                                                                "mesh_select_through");

and

  const bool use_zbuf = !(mesh_select_through || XRAY_FLAG_ENABLED(vc->v3d));

and

if (ts->selectmode & SCE_SELECT_EDGE && !mesh_select_through) {
      mesh_foreachScreenEdge_clip_bb_segment(
          vc, do_mesh_box_select__doSelectEdge_pass0, &cb_data, clip_flag);
}

and

  if (ts->selectmode & SCE_SELECT_FACE) {
    if (mesh_select_through) {     
    mesh_foreachScreenFaceVerts(
            vc, do_mesh_box_select__doSelectFace, &data, V3D_PROJ_TEST_CLIP_NEAR | V3D_PROJ_TEST_CLIP_BB);
    }
    else {
       if (use_zbuf) {
        data.is_changed |= edbm_backbuf_check_and_select_faces(
            esel, vc->depsgraph, vc->obedit, vc->em, sel_op);
      }
      /* Xray Mode with face center selection */
      else {
        mesh_foreachScreenFaceCenter(
            vc, do_mesh_box_select__doSelectFaceCenter, &data, V3D_PROJ_TEST_CLIP_DEFAULT);
      }
    }
  }

And at least it builds, but it will crash when trying to box select.

The worst part is I don’t know why, other than something done here.

And apparently no crash log on windows, hilarious.

https://docs.blender.org/manual/en/latest/troubleshooting/crash.html

Thanks for helping. There’s probably reasons to not do this, and other ways around it, but if anybody could please point me in a direction so I can become competent with dealing with this type of situation I would be grateful.

I wonder if I could just do something else entirely like getting a Tool Setting inside of the keymap?

Answer 1

Ok finally figured it out.

What I needed was to know if a boolean operator property was true or false.

To do that, I did this:

  PropertyRNA *prop = RNA_boolean_get(op->ptr, "name_of_property");

Before, I was doing 'RNA_struct_find_property' which will only see if the property exists. It will return true regardless of the boolean being true or false. Not useful for my problem.

To figure out why Blender was crashing, use a debugger instead of building and just hoping for the best like I was doing ;)

For blender, follow this guide https://wiki.blender.org/wiki/Tools/Debugging/Python_Visual_Studio

Mine was crashing because I was using 'op' without initializing it. To do that, I put it here:

static bool do_mesh_box_select(ViewContext *vc,
                               wmGenericUserData *wm_userdata,
                               const rcti *rect,
                               const eSelectOp sel_op,
                               wmOperator *op)